From nobody Thu Jan 27 17:13:59 2022 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 81B7F199562F; Thu, 27 Jan 2022 17:14:01 +0000 (UTC) (envelope-from glebius@freebsd.org) Received: from cell.glebi.us (glebi.us [162.251.186.162]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "cell.glebi.us", Issuer "cell.glebi.us" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Jl6d44G4Rz4dZm; Thu, 27 Jan 2022 17:14:00 +0000 (UTC) (envelope-from glebius@freebsd.org) Received: from cell.glebi.us (localhost [127.0.0.1]) by cell.glebi.us (8.16.1/8.16.1) with ESMTPS id 20RHDxaM049214 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Thu, 27 Jan 2022 09:13:59 -0800 (PST) (envelope-from glebius@freebsd.org) Received: (from glebius@localhost) by cell.glebi.us (8.16.1/8.16.1/Submit) id 20RHDxBo049213; Thu, 27 Jan 2022 09:13:59 -0800 (PST) (envelope-from glebius@freebsd.org) X-Authentication-Warning: cell.glebi.us: glebius set sender to glebius@freebsd.org using -f Date: Thu, 27 Jan 2022 09:13:59 -0800 From: Gleb Smirnoff To: Mateusz Guzik Cc: src-committers@freebsd.org, dev-commits-src-all@freebsd.org, dev-commits-src-main@freebsd.org Subject: Re: git: e1882428dcbb - main - ifnet/mbuf: provide KPI to serialize/restore m->m_pkthdr.rcvif Message-ID: References: <202201270600.20R602KH059925@gitrepo.freebsd.org> List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-main@freebsd.org X-BeenThere: dev-commits-src-main@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: 4Jl6d44G4Rz4dZm X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=softfail (mx1.freebsd.org: 162.251.186.162 is neither permitted nor denied by domain of glebius@freebsd.org) smtp.mailfrom=glebius@freebsd.org X-Spamd-Result: default: False [-3.10 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FREEFALL_USER(0.00)[glebius]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; MIME_GOOD(-0.10)[text/plain]; HAS_XAW(0.00)[]; DMARC_NA(0.00)[freebsd.org]; R_SPF_SOFTFAIL(0.00)[~all:c]; NEURAL_HAM_LONG(-1.00)[-1.000]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-1.00)[-0.999]; MLMMJ_DEST(0.00)[dev-commits-src-all,dev-commits-src-main]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:27348, ipnet:162.251.186.0/24, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[] X-ThisMailContainsUnwantedMimeParts: N On Thu, Jan 27, 2022 at 09:11:01AM -0800, Gleb Smirnoff wrote: T> Mateusz, T> T> On Thu, Jan 27, 2022 at 05:59:03PM +0100, Mateusz Guzik wrote: T> M> --- trap 0xc, rip = 0xffffffff80616d98, rsp = 0xfffffe06462889e0, rbp T> M> = 0xfffffe06462889e0 --- T> M> m_rcvif_serialize() at m_rcvif_serialize+0x8/frame 0xfffffe06462889e0 T> M> netisr_queue_src() at netisr_queue_src+0x15c/frame 0xfffffe0646288a30 T> M> route_output() at route_output+0x890/frame 0xfffffe0646288c70 T> M> sosend_generic() at sosend_generic+0x456/frame 0xfffffe0646288d10 T> M> soo_write() at soo_write+0x43/frame 0xfffffe0646288d40 T> M> dofilewrite() at dofilewrite+0x81/frame 0xfffffe0646288d90 T> M> sys_write() at sys_write+0xbc/frame 0xfffffe0646288e00 T> M> amd64_syscall() at amd64_syscall+0x107/frame 0xfffffe0646288f30 T> M> fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0646288f30 T> M> --- syscall (4, FreeBSD ELF64, sys_write), rip = 0x3b000ba6703a, rsp = T> T> The breaking commit is 6871de9363e559fef6765f0e49acc47f77544999, T> not exactly this one. T> T> This means route_output() tries to queue on netisr an mbuf without T> rcvif set. And by means of route protocol it is legitimate. T> T> The question is how did that work before? T> T> If we look at netisr.c before my change, we see that on dequeuing we T> would require the rcvif for VIMAGE case: T> T> netisr_process_workstream_proto(): T> T> VNET_ASSERT(m->m_pkthdr.rcvif != NULL, T> ("%s:%d rcvif == NULL: m=%p", __func__, __LINE__, m)); T> CURVNET_SET(m->m_pkthdr.rcvif->if_vnet); T> netisr_proto[proto].np_handler(m); T> T> This won't panic without VIMAGE however. T> T> It also is dereferenced in netisr_drain_proto_vnet(), but apparently T> this function is also disabled without VIMAGE. Oh, here is the answer: static void rt_dispatch(struct mbuf *m, sa_family_t saf) { struct m_tag *tag; /* * Preserve the family from the sockaddr, if any, in an m_tag for * use when injecting the mbuf into the routing socket buffer from * the netisr. */ if (saf != AF_UNSPEC) { tag = m_tag_get(PACKET_TAG_RTSOCKFAM, sizeof(unsigned short), M_NOWAIT); if (tag == NULL) { m_freem(m); return; } *(unsigned short *)(tag + 1) = saf; m_tag_prepend(m, tag); } #ifdef VIMAGE if (V_loif) m->m_pkthdr.rcvif = V_loif; else { m_freem(m); return; } #endif netisr_queue(NETISR_ROUTE, m); /* mbuf is free'd on failure. */ } Give me 15 more minutes, I will either revert the change or make a hot fix. -- Gleb Smirnoff