From nobody Thu Mar 10 14:00:25 2022 X-Original-To: freebsd-net@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 7556C19F161B for ; Thu, 10 Mar 2022 14:00:30 +0000 (UTC) (envelope-from bz@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4KDrLQ2XXTz3Grt; Thu, 10 Mar 2022 14:00:30 +0000 (UTC) (envelope-from bz@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1646920830; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=wuW2Zk9ru+7/oxmuB7epVzPEqk+MbiaSZNw0q0diaeI=; b=Ce7ObnIBQSkJ0lIWm+e78kQdxMdQcgNTpcxkOE7hgmw7rF7OgvKHFYlmrSs/HMXfwjKurf L5m29KbbJ98+B6s/2hDQS1Up8oDqgem4duQxxNa2hLaBXpJ8zwx6+81FNlbMVabFmkR5M4 A9iV9lhZQP9GnsH9cncWm5DiCc0W5mi2xt4tvHz4PzpwFnEHabk5qVRkWDDKAJh7WmY2Zy v9bivNH9NDVo3Gb7OTImM4wsEw/6Ux39aGhmRexILDRttitBDZxfaZeB5NeA8uLx9E1QOH M/nClbZUfIUEgnI0XgV164a6a6UUk3TdU56QKRV3T6jr75ABKeqT183nSn3/gw== Received: from mx1.sbone.de (cross.sbone.de [195.201.62.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx1.sbone.de", Issuer "SBone.DE" (not verified)) (Authenticated sender: bz/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id F401BF6F4; Thu, 10 Mar 2022 14:00:29 +0000 (UTC) (envelope-from bz@freebsd.org) Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:31::2013:587]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.sbone.de (Postfix) with ESMTPS id 895338D4A15D; Thu, 10 Mar 2022 14:00:28 +0000 (UTC) Received: from content-filter.sbone.de (content-filter.sbone.de [IPv6:fde9:577b:c1a9:31::2013:2742]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPS id 9C60CE70814; Thu, 10 Mar 2022 14:00:27 +0000 (UTC) X-Virus-Scanned: amavisd-new at sbone.de Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:31::2013:587]) by content-filter.sbone.de (content-filter.sbone.de [fde9:577b:c1a9:31::2013:2742]) (amavisd-new, port 10024) with ESMTP id NS_kbI5e-tR0; Thu, 10 Mar 2022 14:00:26 +0000 (UTC) Received: from nv.sbone.de (nv.sbone.de [IPv6:fde9:577b:c1a9:31::2013:138]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPSA id B848DE707BB; Thu, 10 Mar 2022 14:00:25 +0000 (UTC) Date: Thu, 10 Mar 2022 14:00:25 +0000 (UTC) From: "Bjoern A. Zeeb" To: Wolfgang Zenker cc: Kristof Provost , Johan Hendriks , mops@punkt.de, FreeBSD Net Subject: Re: epair and vnet jail loose connection. In-Reply-To: Message-ID: References: <051d51b6-2a07-fbc6-7b4d-13947e7fcdbb@gmail.com> <65a18f1b-ea22-a3d2-b4ad-41fd52b7fbae@gmail.com> X-OpenPGP-Key-Id: 0x14003F198FEFA3E77207EE8D2B58B8F83CCF1842 List-Id: Networking and TCP/IP with FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-net List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-net@freebsd.org MIME-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="0-287681838-1646920525=:68830" Content-ID: ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1646920830; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=wuW2Zk9ru+7/oxmuB7epVzPEqk+MbiaSZNw0q0diaeI=; b=eId/1uPVJ4bCopbfGkw0sBUbWZ8M17l5eE2RMIjJNHgsR8JPzUz9wiBuEcl+vo1+opwK3r aECxH2x5d1rFKd/TWCT76O8LguTYoRFyhkj5KnHU8mPfoPL4GxCc5zhcatOZzEdAOFiSC8 nBrsA2E8rjaijagdNlyII2eOaEaHJOLPKIRG0RRreRP1d2rUX8nxaIr8AgzJCKQUbcMdYY QTuSK/E950TF7dpwT0Lv0wVoZWL31OL7ROufozTgyGXMTZyFqZCNLkgxvSW8wiu8coBiXm AEcngcMieTK4pPjhvrRMxhfW5i9nZFRnFMjIjltPXQH+AKLFhemlNtE29RcfAQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1646920830; a=rsa-sha256; cv=none; b=upgE4KR7lNzSiKrk2GVIvwOUNC9/7mf/UZ1Ymwl/7f9WQK0XVteTUqWJZruvQmNKoPcsCL XahSspcFPOMjn62GLKEAtGEH+2A6su7oWQtinWTDRdiouL+xjPyvj2DslQnL6hf4HK91H1 ef290LX4N8T0t8EotgaHoPn3Q4lkPnuh1Fsmu0gs7Wzns8UgJyjk1rX5ov5XGPqYgM+ElO RTIh+Dnz3LfilMjrPEQcsnOn3hvpZ4QKGrfhOlJIetSHWRizwtqbZQ0HtEOW3Dp7xFYX+X OPZ+E0jj4MSCGHXHpzXLcDqb7b46Y1bZ5PvsJ9V8Re+bVjpodYQjvvgE7JgEnQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --0-287681838-1646920525=:68830 Content-Type: TEXT/PLAIN; CHARSET=UTF-8; FORMAT=flowed Content-Transfer-Encoding: 8BIT Content-ID: On Thu, 10 Mar 2022, Wolfgang Zenker wrote: Hi, >>> I did do a  hey -h2 -n 10 -c 10 -z 60s https://wp.test.nl to that machine and in the 60 seconds the jail became unresponsive. Then i did run the dtrace.sh script above like so /root/bin/dtrace.sh > /root/dtrace_output >>> >>> I hope this helps, if you need anything please let me know. Also root access is possible if you want. That way you do not have to create a test environment. > >> Were there other epair interfaces running at this time, with active traffic? > >> The dtrace output appears to show that the appropriate callouts (to epair_tx_start_deferred()) are getting through, so I’d expect traffic to be flowing. > > There is one second jail using epair on that system, using the same > bridge as well. This second jail is a low-traffic system, it is unlikely > but possible that there was some traffic during that time. Were you bridging or routing? I seem to remember if_bridge being loaded from loader? So you'll always have some broad-/multi-cast. > In all previous cases this second jail continued to be reachable all > the time. I don't know the latest incarnations of epair code very well anymore. I'd probably go and look at stats (netstat etc) for the interface and possibly protocols as well before restarting the jail; check if there are packets queued, dropped exacessively or if the in/out packet coutners are still increasing (on both sides)? I'd probably also run a tcpdump then on both sides of the epair to see if packets are still arriving on one side and not the other? And if it is a bridging setup, I wonder if taking that out of the picture (you could remove the epair end from the bridge, put an address on it and send a ff02::1 mc ping6 or something) and see. Also does setting the epairs down/up (on both ends) make any differences? I am basically trying to narrow things down, as restarting entire jails and with that a network stack is a lot more changes than just epair. /bz -- Bjoern A. Zeeb r15:7 --0-287681838-1646920525=:68830--