From nobody Tue May 23 21:15:33 2023 X-Original-To: freebsd-net@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4QQnFm1RlTz4T42T for ; Tue, 23 May 2023 21:17:16 +0000 (UTC) (envelope-from zec@fer.hr) Received: from EUR05-VI1-obe.outbound.protection.outlook.com (mail-vi1eur05on2058.outbound.protection.outlook.com [40.107.21.58]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "DigiCert Cloud Services CA-1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4QQnFl60wnz3xGs for ; Tue, 23 May 2023 21:17:15 +0000 (UTC) (envelope-from zec@fer.hr) Authentication-Results: mx1.freebsd.org; none ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=dWPwLfgvef/6W+d7Z/a8X0+A+AAHpIT3cFDgSkTjC/X/neRdzRPRQTBux1mixTylhcYuI2xWYkh6gibXzcep2Hx9fqJzka/W6Qt15kXT+WkQRiEnMbT2SPj8nFgMtggo7QuasnoZkMHx3bm9iCVMudbK2taIAsBXrTH/CrXGyt1Du0+lmE4PjyvHDgkvAavS9Nxi5lnq96eH1szv2uezP5PJIVXlygGsGlRwRJs3DiDPMoQFn825lxSIyIZxssxFPC8q9zPAXtVUmZwDs671+0tsC8KBywrUzdFwVpffFyowboMcgqSU8m3i8EN8XsUx29DVTT/H4EJScU2NEuJGNQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=y+HEELBHP/Cxptc8PDKbFrWZ+hIvkHkCa9asNEeV58E=; b=d68GCfqt2PHgnKcE4tiYAZb0WT0OdggXJZSY38093T8Bpct0KrRSbDIed5BkvdXjUh4Iv4HWgZUDCq9KJ358YA3ZXFAPbehfDVv8OoEcZpYSbxd2AwQFa8gZcXQhCKeASkou+uGCnIiubKOXWvclTkPD3eTevxf4G9BSxvhcZM1J56U+051yepjsls6pvhMK6FzzuygTsOuTPCPLkoQg84NtNz7In+IfzIyoMd8OHCSonMmXtaat86Ac7i301cfEbtjlsGZpcMrzuyf1vZ0Jt//mVYi0SVLPFGnR5H7JBB3SoOIXXgqjbf45VHx2IpkDzgmPL3ALjj3sqThonX4vQA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=fer.hr; dmarc=pass action=none header.from=fer.hr; dkim=pass header.d=fer.hr; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ferhr.onmicrosoft.com; s=selector2-ferhr-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=y+HEELBHP/Cxptc8PDKbFrWZ+hIvkHkCa9asNEeV58E=; b=JujVejyVMuIkG4qAVKKutKM+scn452VgXNpunXS5DVcOkfgFoZ0ul2tj25xXUl04YycYOfbLT/i5RvnLknKTf0YGqll2pmV7NsfJ77PxWFxrv3zr9lvOrsXXp15eMhHlY4LlkKrhMIeFHb895wAGIq2kRzGMrh5jWjKGjuo4MuE= Received: from VE1PR08MB4783.eurprd08.prod.outlook.com (2603:10a6:802:a9::16) by GV1PR08MB7345.eurprd08.prod.outlook.com (2603:10a6:150:20::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6411.28; Tue, 23 May 2023 21:17:09 +0000 Received: from VE1PR08MB4783.eurprd08.prod.outlook.com ([fe80::89c9:9a19:d24d:44ad]) by VE1PR08MB4783.eurprd08.prod.outlook.com ([fe80::89c9:9a19:d24d:44ad%3]) with mapi id 15.20.6411.029; Tue, 23 May 2023 21:17:09 +0000 Date: Tue, 23 May 2023 23:15:33 +0200 From: Marko Zec To: Benoit Chesneau Cc: "freebsd-net@FreeBSD.org" Subject: Re: how to increase the vnet speed? Message-ID: <20230523231533.32c1d022@x25> In-Reply-To: References: X-Mailer: Claws Mail 3.19.0 (GTK+ 2.24.33; amd64-portbld-freebsd13.1) Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-ClientProxiedBy: VI1P193CA0001.EURP193.PROD.OUTLOOK.COM (2603:10a6:800:bd::11) To VE1PR08MB4783.eurprd08.prod.outlook.com (2603:10a6:802:a9::16) List-Id: Networking and TCP/IP with FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-net List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-net@freebsd.org MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: VE1PR08MB4783:EE_|GV1PR08MB7345:EE_ X-MS-Office365-Filtering-Correlation-Id: a46fd69f-a06e-4588-dcf8-08db5bd31145 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: IGOtCdWVKo5lPRFqB1XxY8CfdTmkR0iLI2dicb868qeLgsbrHF2cYq9xnL9bFODms8ps/WpvuCslaGyFeE9qWMgpSbLkXi0OSryOViWmzMpIr5nHVQX30gK8t3AL/DtdsPJRPbPpmjDzwgB5GAh/CD0I1BLTvxzPqiQgfYApE/Uaibqt9XsyXnJndGUMJx+OGlxE52DokYTd0S62YZQosDMzP3fRe7VfJHq9UbA0Z1mRLgEQ75xV7CHmjjxSyNDrjqyWAXUj5Zkej8L7FfBrZxMwgeZ+oQkbjdgj6TrjP1CdnEHHffwf0kmGesTDNNyOmatpLRIxEIK1wA+PUdn9okfuX3RonHUEy0UsRClC68TAoqsGpx4320qIjgRqI72hHhbS9ZBxm/XvG5IEsZUd7LPUFEjX5HmDJ/mpcKu8i6Iz9ww7JghWd1N6kyjCqD5IBH+g7UkQqwRXctTj8rm3OEJtpSwLeDCP8t+bB750A9hG/Xz5O47SOAXAeGfSVBJuV3IWlhBZGFD/KZ2rJ7ezne+2dfFinFkbH31QBtIunPAngs5lRP2Btcf2LxcVoRYZ X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VE1PR08MB4783.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230028)(7916004)(39860400002)(376002)(366004)(346002)(396003)(136003)(451199021)(5660300002)(8676002)(8936002)(86362001)(2906002)(55236004)(6506007)(9686003)(6512007)(1076003)(26005)(41320700001)(4744005)(186003)(4326008)(66556008)(66476007)(66946007)(6916009)(33716001)(316002)(786003)(38100700002)(478600001)(6486002)(41300700001)(6666004);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?J2X/crDMiObea5KfY02lVQsAVUYcrn4/UfXZBVh7vZX0nlOZxurlRRLVfKpV?= =?us-ascii?Q?ea/UHLPnZrR9D4fJK7p1Ui8YXoMoo5kQbos+ruZTP3pxvQDrA14YpCAWdZGX?= =?us-ascii?Q?vhp+43gspWyjAuEF9QEp0nZZUQAiYvxeFFUz46DWwL+/GI6alBTtu8OD2eXf?= =?us-ascii?Q?U+D6TdAjz5mvTJSVeKKr+0t1Mnk/bmPde56V1YXAiBFJMVdUlFJ+Ir0UFzRh?= =?us-ascii?Q?uIZjmfKbtKFgqLsxLT8r52dGm4zgQwpvDZkDt7DFdM1QIhHX2ilwYuZUJ5Uf?= =?us-ascii?Q?gaFp/Ixxgznded7L5dl08y62lQCQPtC++L0DEmXvAyBxG06SPyH7VUem0Ydp?= =?us-ascii?Q?lcMdW0P0Fjsip6tz06zKo6XBIECxlwMsggi+BoUBRaeeavRt8sgZqIcYyQku?= =?us-ascii?Q?DelFo3IZAyUG2I5OsX+rRpctMIsyq8VQeJaVgvhxqoFToRWINUDBVMlsreya?= =?us-ascii?Q?n0DEP85hALGz0k58If18BdR6v9RUCVfOpIO/8WhkBeK/ziZjx1W5dYa8S//6?= =?us-ascii?Q?ZoQu5WJaD0sQk6Fuz/ZyJCW6+u5ECQPTShCPlyQ9IJix74dobj4TcJ2nsnnh?= =?us-ascii?Q?EaLoU/AKSa0ff6wZA1wDwxjQHaweJOfO0yWaHuzo8LfGJC2J0CM6FcU5p6Cu?= =?us-ascii?Q?14tbHbUbKY60IjQDqk7W49wEh405RKdsffhPToHshmY1uE2bx+QjLjtsZ1fy?= =?us-ascii?Q?FgtqPRi3CqekSSdqOlHC5Czt7fIoK6FVWPEO11JUDMbfnk/iL83FMILjn6So?= =?us-ascii?Q?OmzW/bq6iDStp52p+Gra56HftUgzV8hnMNRenP/D5EnSF0jLgEtHOP4JzY+y?= =?us-ascii?Q?KC+izbzUjHZd6q6sva7c+IpoOWZ4hiovtl8D6AbvbEIW2NWOUmOz8Q/OLsOs?= =?us-ascii?Q?4m2l2L59yGfNDnvoXN4mrn51tyMHNEHT1hwRyzldzOIS+4waeRAEbCc7qcuu?= =?us-ascii?Q?RWPwY6VpfxNML3SE2wenkwZa8UEGRlelBRVBQzQq1/M8/lPlRxk1PvB8rYlK?= =?us-ascii?Q?lgH/zKBQ+xfsMxsWAWd2nzftGNIa+PGIZ5bq1tzJOAGSrT7+odk4pe7ft1r3?= =?us-ascii?Q?u08Np/t2VJX3NdTD7TdY+gVGVxtuoG7SRxg6QF7Upr2FQezQOZfvxFpy7co7?= =?us-ascii?Q?Mt7Be6T2hIwKThHVUx68JNqm+TOWPbRCybFfn4OFhuu1A3Lo38cnET6416lY?= =?us-ascii?Q?PwKZBIr6Lt8HUVT0vumynUa3+o7R2R1hMyd/WnKG94ymaW6gBShPJCrNVVNR?= =?us-ascii?Q?V1NkY8rYS3zgk84d1ngdUgIInhHQoOJ6zCjvBmqkQsZedOehxMFvRmqFiYx4?= =?us-ascii?Q?4FWMTFm5/rczsBs3gQhSmTJTQ7RbKmTOjNArhYbbAepkxj4ak2LgDeSFcYlx?= =?us-ascii?Q?r6c5YYjyJTdGXSBTMBYIzDiVkv3NlVkznWsBfaU0dna4/kw/mnT9Cu11fiIs?= =?us-ascii?Q?AlMaqKPNhmnu1tdxVqlWVXNU5aWlE7/alXmKn25tlsLKCvYIsZAcBGjyk7Kj?= =?us-ascii?Q?NTmNdufhvq5QMx2gMEerilC4UpiFRm//teJ9Ru5yrfbuFguLtzT2Pb91TP3A?= =?us-ascii?Q?VJyI6aQC77GRqopLfnQ=3D?= X-OriginatorOrg: fer.hr X-MS-Exchange-CrossTenant-Network-Message-Id: a46fd69f-a06e-4588-dcf8-08db5bd31145 X-MS-Exchange-CrossTenant-AuthSource: VE1PR08MB4783.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 May 2023 21:17:09.0638 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: ca71eddc-cc7b-4e5b-95bd-55b658e696be X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: c+oheC9TjvPejKeXlp+2RgSwD7V8fumoDBHlaHLv3Iz8J6vF80Aibt7Pv6vshsGf X-MS-Exchange-Transport-CrossTenantHeadersStamped: GV1PR08MB7345 X-Rspamd-Queue-Id: 4QQnFl60wnz3xGs X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:8075, ipnet:40.104.0.0/14, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N On Tue, 23 May 2023 19:58:07 +0000 Benoit Chesneau wrote: > Hi all, > > I've created a jail using bastille and setup network. The mainin > terface is a 25Gbps nic and between hosts I get 24.6 Gbits/sec : [...] > But between one host and the jail I only get 3.96 Gbits/sec [...] > Is there a way to increase the performance of the of the jail? The > nice is a mellannox ConnectX-4 Lx, mce(4) . Modern NICs offload a lot of the protocol stack processing (checksum, segmentation, and / or reassembly) from the CPU to dedicated silicon, whereas inter-vnet traffic needs to be handled completely in software, that's where the difference comes from. Perhaps we could gain some speed by abusing mbuf flags to skip RXCSUM for epair traffic, maybe even skip and fake TXCSUM... Marko