From nobody Fri Jan 14 23:34:23 2022 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id E0B4A1966DCD for ; Fri, 14 Jan 2022 23:34:35 +0000 (UTC) (envelope-from jason@tubnor.net) Received: from mail.tubnor.net (mail.tubnor.net [103.236.162.16]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA512) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4JbHhB3Lbtz3hRT for ; Fri, 14 Jan 2022 23:34:34 +0000 (UTC) (envelope-from jason@tubnor.net) Received: from THEMASTER (119-18-29-78.77121d.mel.static.aussiebb.net [119.18.29.78]) by mel01.ar18.net (OpenSMTPD) with ESMTPSA id 4fe4f894 (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256:NO); Sat, 15 Jan 2022 10:34:22 +1100 (AEDT) From: To: "'Daniel Braniss'" , References: <878640B2-CB46-4066-8655-20A18D3A6A7D@cs.huji.ac.il> In-Reply-To: <878640B2-CB46-4066-8655-20A18D3A6A7D@cs.huji.ac.il> Subject: RE: bhyve terribly slow when using tagged vlan Date: Sat, 15 Jan 2022 10:34:23 +1100 Message-ID: <02d301d8099f$430c01a0$c92404e0$@tubnor.net> List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Outlook 16.0 Thread-Index: AQGIO0EJLy8AzF3WPPlJhxOjHlv/eK0Cx28g Content-Language: en-au X-Rspamd-Queue-Id: 4JbHhB3Lbtz3hRT X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of jason@tubnor.net designates 103.236.162.16 as permitted sender) smtp.mailfrom=jason@tubnor.net X-Spamd-Result: default: False [-2.70 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; SUBJECT_ENDS_SPACES(0.50)[]; FREEFALL_USER(0.00)[jason]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:103.236.162.16]; MID_RHS_MATCH_FROM(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; DMARC_NA(0.00)[tubnor.net]; ARC_NA(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCPT_COUNT_TWO(0.00)[2]; FROM_NO_DN(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; MLMMJ_DEST(0.00)[freebsd-virtualization]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:~]; ASN(0.00)[asn:133159, ipnet:103.236.162.0/23, country:AU]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[] X-ThisMailContainsUnwantedMimeParts: N Try: ifconfig ixl1 -lro -tso Observations that these options active on a stacked VLAN interface causes issues. Typically when a VLAN interface is pushed into a bridge, these should be stripped, but are not. Cheers, Jason. > -----Original Message----- > From: owner-freebsd-virtualization@freebsd.org virtualization@freebsd.org> On Behalf Of Daniel Braniss > Sent: Friday, 14 January 2022 9:50 PM > To: freebsd-virtualization@freebsd.org > Subject: bhyve terribly slow when using tagged vlan > > hi, > the host is a very new Dell PowerEdge R6525 running FreeBSD 13.0, I have > connected ixl0 to an untagged link and ixl1 to a tagged link > > I have created 2 bhyve clients, identical, except for the links, one is using ixl0 > and the other ixl1.340 > > the untagged boots diskless in about 40 seconds, while the one using the > tagged one takes 12 minutes! > > I compared their dmesgs and they are identical (except for minor diffs like > mac address) > > any ideas? > > danny > > some more data: > > sa# uname -a > FreeBSD sa 13.0-STABLE FreeBSD 13.0-STABLE #4 huji-13-736697896: Mon Dec > 13 14:03:04 IST 2021 danny@hms-00:/home/obj/hms- > 00/h/rnd/git/stable/13/amd64.amd64/sys/HUJI amd64 > sa# ifconfig > ixl0: flags=8963 > metric 0 mtu 1500 > > options=4a500b9 VLAN_HWCSUM,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,NOMAP> > ether b4:96:91:97:08:14 > inet 132.65.116.32 netmask 0xfffff000 broadcast 132.65.127.255 > media: Ethernet autoselect (10Gbase-T ) > status: active > nd6 options=29 > ixl1: flags=8963 > metric 0 mtu 1500 > > options=4e507bb O_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,R > XCSUM_IPV6,TXCSUM_IPV6,NOMAP> > ether b4:96:91:97:08:15 > media: Ethernet autoselect (10Gbase-T ) > status: active > nd6 options=29 > bge0: flags=8802 metric 0 mtu 1500 > > options=c019b WCSUM,TSO4,VLAN_HWTSO,LINKSTATE> > ether b0:7b:25:dd:ff:9c > media: Ethernet autoselect > nd6 options=29 > bge1: flags=8802 metric 0 mtu 1500 > > options=c019b WCSUM,TSO4,VLAN_HWTSO,LINKSTATE> > ether b0:7b:25:dd:ff:9d > media: Ethernet autoselect > nd6 options=29 > lo0: flags=8049 metric 0 mtu 16384 > > options=680003 6> > inet6 ::1 prefixlen 128 > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5 > inet 127.0.0.1 netmask 0xff000000 > groups: lo > nd6 options=21 > ixl1.340: > flags=8943 > metric 0 mtu 1500 > options=4200401 > ether b4:96:91:97:08:15 > inet 132.65.80.251 netmask 0xfffff000 broadcast 132.65.95.255 > groups: vlan > vlan: 340 vlanproto: 802.1q vlanpcp: 0 parent interface: ixl1 > media: Ethernet autoselect (10Gbase-T ) > status: active > nd6 options=29 > ue0: flags=8802 metric 0 mtu 1500 > ether b0:7b:25:e5:dc:57 > nd6 options=29 > vm-servers: flags=8843 > metric 0 mtu 1500 > ether 0a:91:de:0e:f6:26 > id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 > maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 > root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 > member: tap0 flags=143 > ifmaxaddr 0 port 10 priority 128 path cost 2000000 > member: ixl0 flags=143 > ifmaxaddr 0 port 1 priority 128 path cost 2000 > groups: bridge vm-switch viid-d5539@ > nd6 options=9 > vm-system: flags=8843 > metric 0 mtu 1500 > ether 42:5b:f8:8f:72:00 > id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 > maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 > root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 > member: tap1 flags=143 > ifmaxaddr 0 port 11 priority 128 path cost 2000000 > member: ixl1.340 flags=143 > ifmaxaddr 0 port 6 priority 128 path cost 2000 > groups: bridge vm-switch viid-54b53@ > nd6 options=9 > tap0: > flags=8943 > metric 0 mtu 1500 > description: vmnet-bhv-08-0-servers > options=80000 > ether 58:9c:fc:10:82:13 > groups: tap vm-port > media: Ethernet autoselect > status: active > nd6 options=29 > Opened by PID 9357 > tap1: > flags=8943 > metric 0 mtu 1500 > description: vmnet-bhv-09-0-system > options=80000 > ether 58:9c:fc:10:ff:a2 > groups: tap vm-port > media: Ethernet autoselect > status: active > nd6 options=29