From nobody Fri Sep 13 07:03:56 2024 X-Original-To: freebsd-net@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4X4lf91Y74z5W8cx for ; Fri, 13 Sep 2024 07:04:01 +0000 (UTC) (envelope-from cryintothebluesky@gmail.com) Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com [IPv6:2a00:1450:4864:20::331]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4X4lf86gN2z4P9F for ; Fri, 13 Sep 2024 07:04:00 +0000 (UTC) (envelope-from cryintothebluesky@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-wm1-x331.google.com with SMTP id 5b1f17b1804b1-42cc43454d5so4852695e9.3 for ; Fri, 13 Sep 2024 00:04:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1726211039; x=1726815839; darn=freebsd.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=BIyGYmYiFRoIHYNRYqJeC/DHp1kUJ3rlr7O+EU9Hv0g=; b=EmCKHzg4bEGE9xDcVOhphCfTMFwuF0/AZDgD8mhy2pwj6uZY8YPOx+tyVuyeJBCXvw DF6qwhE1itO6uFiauptZy+UunbPEHptSI2EtB+7G6HtvuNzBsNU9ep6fxXRUSAFfhg8O k7sscWJJ1cSsPm99+1wfdNUdOLCijjIYtg9aLJJrGZhxgZ0+oJsfwwC3rK6vuZtmO8/o DK89uFLa9NXHz4SuhNZOjCWIx2Aku3jhDqoqrczOKCqgvinFXoOxgqrtUyOdrT26oIVn JMtTkc02dFFCrRlNepmMv00GPigSA0pe4IEu+9/Q2sSnn94jIEpAHqnCG1i1WQy97hTa mqFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726211039; x=1726815839; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BIyGYmYiFRoIHYNRYqJeC/DHp1kUJ3rlr7O+EU9Hv0g=; b=e/i3XXtKFYvlqu5eXoDH4A6mq2m06Da4HnSHOydsr6WBhqhaYfkQngjySlB24XcAuU 2qEPiC/FdGN+JtO3kgaLcYq9eC7LrwIMBdsrby/uIvlcbmYirHjHm8lXIC1QGEA1atu1 NFQf/13UO+aQRdJAmef5VM3CHRiXN2GuR3c4iiC1gkTsES723bn5n+LwsvFLzBmq4O9X yYZr/i9xkoyhIst6AU0k3+HSXO0m4E5C/PzHO+DPduxKUKIcsbrd5jIqSrAxkEWNDS1o 1bL8vOhb5BgzN3bdBiDhEczfZZzce28KSUWHG665zZpJvSIKx9hcfztdldeUDej4BxNC 5qvQ== X-Gm-Message-State: AOJu0Yy01CSzskkxUsOTnlYXEhqYQ5vEtEMFDebb65xAybp84YC0erRR T8/g9Dm1a9a9vydKgeN5ekIRzqFk08j17wgesJRIBe0HgBKM+bZ/HDawKw== X-Google-Smtp-Source: AGHT+IGRPNcGv7thhqTa8mOOK6FMIZm/nBhUMJEf9qN7ytczfJqnh2I5Vh+7s0YdSjWAThfVvkVWJg== X-Received: by 2002:a05:600c:654e:b0:42c:e0da:f132 with SMTP id 5b1f17b1804b1-42d9a9109fdmr11888355e9.32.1726211038392; Fri, 13 Sep 2024 00:03:58 -0700 (PDT) Received: from z600.home.lan (29.129.159.143.dyn.plus.net. [143.159.129.29]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-42d9b05a76csm13760295e9.8.2024.09.13.00.03.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Sep 2024 00:03:57 -0700 (PDT) Date: Fri, 13 Sep 2024 08:03:56 +0100 From: Sad Clouds To: Paul Procacci Cc: freebsd-net@freebsd.org Subject: Re: Performance issues with vnet jails + epair + bridge Message-Id: <20240913080356.98ea2c352595ae0bbd9f0ce8@gmail.com> In-Reply-To: References: <20240912181618.7895d10ad5ff2ebae9883192@gmail.com> X-Mailer: Sylpheed 3.8.0beta1 (GTK+ 2.24.33; x86_64-pc-linux-gnu) List-Id: Networking and TCP/IP with FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-net List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-net@FreeBSD.org Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] X-Rspamd-Queue-Id: 4X4lf86gN2z4P9F I built new kernel with "options RSS" however TCP throughput performance now decreased from 128 MiB/sec to 106 MiB/sec. Looks like the problem has shifted from epair to netisr PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND 12 root -56 - 0B 272K CPU3 3 3:45 100.00% intr{swi1: netisr 0} 11 root 187 ki31 0B 64K RUN 0 9:00 62.41% idle{idle: cpu0} 11 root 187 ki31 0B 64K CPU2 2 9:36 61.23% idle{idle: cpu2} 11 root 187 ki31 0B 64K RUN 1 8:24 55.03% idle{idle: cpu1} 0 root -64 - 0B 656K - 2 0:50 21.50% kernel{epair_task_2} Are there any other tricks I can try to distribute the load over multiple CPU cores? I found this document which describes various vnet examples: https://freebsdfoundation.org/wp-content/uploads/2020/03/Jail-vnet-by-Examples.pdf The problem is, with Raspberry Pi I cannot add any pcie network cards with multiple ports or SR-IOV features. I was hoping to use some software based virtualization device, but if the scalability is this bad, then I may have to discard vnet completely and configure jails to use the host network stack. Thanks.