From nobody Fri May 27 20:17:29 2022 X-Original-To: net@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id BB7851B58DD9 for ; Fri, 27 May 2022 20:17:30 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4L8x1Q4q9Tz3FtT for ; Fri, 27 May 2022 20:17:30 +0000 (UTC) (envelope-from jhb@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1653682650; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=SXvG16u2Th+n/qF4cQ/azqSaYCEW6UZqXOEFG8j1b5I=; b=kJMkUnGCjEgD4/80lfKeVcA2aJ1J/pqYnOy+0bi6B5REiPNBTNXX6Eq9BaQFWImlTJsLTG EleOAMh9cywLSdcavPtL9gLLK8eo6kAz6bNiebX9Ri4pYZguu7FVKpVJ+BzM9yOSaSO0Re S0bsAbpjK/bHut8IdpxAhcqzFC55o1Yr+dNyv0cI9zEgR4mtzFLzWJI7DhDa1iAoiVrWWA Rpf/OowxisL44fabmoAh9fJ9PfIb7Hm+9ZEf89Tdx/b8NtGKtWYwj0jWQWy8gLeUk7PDMS 5QSD46gGF0DCQDSSyAu52MAZ3+MD7dXh6V0usIXuxHGPFI5Z2iV+lLX/VAmd9Q== Received: from [10.0.1.4] (ralph.baldwin.cx [66.234.199.215]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: jhb) by smtp.freebsd.org (Postfix) with ESMTPSA id 551C5AC0D for ; Fri, 27 May 2022 20:17:30 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Message-ID: Date: Fri, 27 May 2022 13:17:29 -0700 List-Id: Networking and TCP/IP with FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-net List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-net@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:91.0) Gecko/20100101 Thunderbird/91.9.0 Content-Language: en-US From: John Baldwin To: net@FreeBSD.org Subject: Adding a new member to m_pkthdr Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1653682650; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=SXvG16u2Th+n/qF4cQ/azqSaYCEW6UZqXOEFG8j1b5I=; b=vsXpW77e0KhFRcInKVRYh9x+KGrnCwixtp6WzzNObLiEG/wP8dMdPqifaq5Rc6baCwxb/Y 42NiLXo5R9cyVNsL0GScPfxcNu18RoK0zLZvtMwprQnvXDeqFcKMqzG99Rz/eGfqaBgM1T x3z8tQFMcvcSbfozJJC4XiN5EO8/z3O25C7LYLkSOQQnTcRqfjGuwDVNRoAzU+UP4+Q523 QcKaqAEFE0JRa+UHIyG9vl67K4AVlKeluno3cr8A/K2fGh8pk0gXMrDwMh/n0L/aGTomE+ GsVV1wpKwKfhl0zgjQ1tlNjhlcdyP/plk2vqas+Ey+CZ7wdjCi/W9nGTbCGqzA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1653682650; a=rsa-sha256; cv=none; b=BAgOIlu4zJ0IbHUgolZq2soohwvdRHilRXsD8Zf69davgcsLTABFo5UlLysNcTxcuJ4c2x KQCCrCK5wfezE7/vPH6Uh8RzABCa0aIc64VFIkqj93RQ+BM+xlHKgXp1dzPDUdcz/zpj9c ZbnOq+Hg05fsDhwsuGTj/eCbXrlhBCI/AT8/U+5ixD8YEBkaIrTZ28Vmoxp5TxngXB3jBm KknR+PMrEyEpaMnbqbV+xHZr8BI3nLCoDk51ZQIX0zECSvTLoWYKEFHOfgKhlVd4II2RR+ laVg9vo62rhQYT3Z1RWnp/DeCoT9L1CRlzl1Byy5Z+jsQKaPvVC4kLBiDoDJNw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N For NIC offload of kernel TLS on the receive side, the kernel needs to know the "leaf" interface that packets arrive on up in the socket buffer layer when appending received packet data to a socket using KTLS. rcvif does not fully work since connections that transit a virtual interface like if_vlan or if_lagg rewrite m_rcvif to be the virtual interface. For KTLS transmit we are able to follow the transmit path down to configure KTLS on the leaf interface. However, while the receive path is usually a mirror of the transmit path, it is not always. In particular, when using a lagg(4) with LACP, the other end of the lagg is free to use whatever hash it chooses to distribute traffic across the lagg ports such that the receive and transmit sides of a connection may transit different ports within a lagg. To provide a leaf interface, I have a patch that adds a new field to m_pkthdr to track the leaf receive interface. It is optional and the only use currently anticipated is KTLS. In the current KTLS patches it is set on received packets by the mlx5 driver. Possibly it could be set more generically in ether_input instead of in individual drivers. It is serialized to an index and generation count while packets are deferred to a netisr similar to rcvif except that it is non-fatal if the ifp cannot be re-materialized when the mbuf is dequeued. Instead, the pointer is simply left as NULL. However, using more space in m_pkthdr is a non-trivial thing, so it's worth raising the conversation more broadly. The change to add this field is in https://reviews.freebsd.org/D35339. Drew has tested this isolated change under load at Netflix and found no impact on performance. -- John Baldwin