From nobody Thu Jun 10 09:50:41 2021 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 0E5ED7CAF32 for ; Thu, 10 Jun 2021 09:50:49 +0000 (UTC) (envelope-from gljennjohn@gmail.com) Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com [IPv6:2a00:1450:4864:20::42d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4G0zkJ2MXQz3r9S for ; Thu, 10 Jun 2021 09:50:48 +0000 (UTC) (envelope-from gljennjohn@gmail.com) Received: by mail-wr1-x42d.google.com with SMTP id c9so1530871wrt.5 for ; Thu, 10 Jun 2021 02:50:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:in-reply-to:references:reply-to :mime-version:content-transfer-encoding; bh=fldPFh7v7t8GMXm2fctdXxG29Tkk9UIB/PC8R6zN+GA=; b=jTCvLZfmNlHTCxiwpXZiAHs16d5FiDuNEWjQzb0HcggZjypx8UnEzt3XSrqFh+eLNv eDx6sSkSf/dIg2YXJeA05CjnJdL+mE/MgHTgowjrmr2oArGMs6kXMTx6/Za4ro6CD1n8 1vo27No3PGA9FLlG2EaRZMcyl+/4Qdz3uUOGwXke6mhldM053eTgPDedxdCFF6uqlMml OMezUyWOn6j9rf5aGFG1Ghw+vK6aMGJ16gndu6n/R638qEB3LZnU+FzkikvCIq0werMh fCXEkkmRew7VdsFc3+xmy8Bl4VYjJKtPQVkSz1g7NGtKrWIuiGCSFkIMhkCqJk8dae+U FrVg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:reply-to:mime-version:content-transfer-encoding; bh=fldPFh7v7t8GMXm2fctdXxG29Tkk9UIB/PC8R6zN+GA=; b=bZQ3VfY5Ann9u+msARU2EkW4bCgwKP5LORM4/p81fImcW0DDkZ2+hcqCpj/oWoviha f9VDXzPaxp/yv1w1GHbIB3xmLnlfk4pDhJ0PMt3kjzq6uV6tKE7vNN64qNxtHdbEi/g0 Z+6iKr+Ts0nH00zx36/4N8Lfou07gH/CAiV8BGHBU3PNXcQcSUUfMgC+23Ql0XYTScDz ag7eOCTF4Qh1/atxZ134/SUu3C6e0ccW8XFzs8btj4uAEjl6N2o1Nsx39pMX8vjEG3Mc +dFT1jY9+NG8e5HTXNky7LbJif99y037+8ocS/h/7WgkzdWPFAgb0dsOfOIpqA5mTFIG v6qw== X-Gm-Message-State: AOAM532gFmrtN1dHqFa1SaZlCLq/PdZ9E17tGNz27I17tPfcDomYdwaQ 97hJ3wU0n0drbTDbawO8aK6hkR4Vchw= X-Google-Smtp-Source: ABdhPJxuw+XhP/9ToaIIMXRHQllhBq91ctY9/igxFkkZTFvDV3gNCY5gEBhNwHrAIF1lbC1gnAIy6g== X-Received: by 2002:a05:6000:188d:: with SMTP id a13mr4396391wri.61.1623318646852; Thu, 10 Jun 2021 02:50:46 -0700 (PDT) Received: from ernst.home (pd9e2360f.dip0.t-ipconnect.de. [217.226.54.15]) by smtp.gmail.com with ESMTPSA id 73sm3141300wrk.17.2021.06.10.02.50.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Jun 2021 02:50:46 -0700 (PDT) Date: Thu, 10 Jun 2021 11:50:41 +0200 From: Gary Jennejohn To: Warner Losh Cc: FreeBSD Current Subject: Re: kernel panic while copying files Message-ID: <20210610095041.38d7597c@ernst.home> In-Reply-To: <20210608155405.5cf0e200@ernst.home> References: <20210607090109.08ecb130@ernst.home> <20210608084646.6a7e1bc7@ernst.home> <20210608155405.5cf0e200@ernst.home> Reply-To: gljennjohn@gmail.com X-Mailer: Claws Mail 3.17.8 (GTK+ 2.24.33; amd64-portbld-freebsd14.0) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4G0zkJ2MXQz3r9S X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=jTCvLZfm; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of gljennjohn@gmail.com designates 2a00:1450:4864:20::42d as permitted sender) smtp.mailfrom=gljennjohn@gmail.com X-Spamd-Result: default: False [-0.18 / 15.00]; HAS_REPLYTO(0.00)[gljennjohn@gmail.com]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; REPLYTO_ADDR_EQ_FROM(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RBL_DBL_DONT_QUERY_IPS(0.00)[2a00:1450:4864:20::42d:from]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; RECEIVED_SPAMHAUS_PBL(0.00)[217.226.54.15:received]; FROM_HAS_DN(0.00)[]; NEURAL_SPAM_SHORT(0.82)[0.818]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; FREEMAIL_REPLYTO(0.00)[gmail.com]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; NEURAL_SPAM_MEDIUM(1.00)[1.000]; SPAMHAUS_ZRD(0.00)[2a00:1450:4864:20::42d:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::42d:from]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-current] X-ThisMailContainsUnwantedMimeParts: N On Tue, 8 Jun 2021 17:54:05 +0200 Gary Jennejohn wrote: [big snip] > Here's the kgdb backtrace with the -O0 kernel: > > (kgdb) bt > #0 0xffffffff8081d706 in doadump (textdump=0) > at /usr/src/sys/kern/kern_shutdown.c:398 > #1 0xffffffff804ef15a in db_dump (dummy=-2138500043, dummy2=false, dummy3=-1, > dummy4=0xfffffe00c62a11b0 "") at /usr/src/sys/ddb/db_command.c:575 > #2 0xffffffff804eef5f in db_command ( > last_cmdp=0xffffffff8114d380 , cmd_table=0x0, dopager=1) > at /usr/src/sys/ddb/db_command.c:482 > #3 0xffffffff804eeb38 in db_command_loop () > at /usr/src/sys/ddb/db_command.c:535 > #4 0xffffffff804f38ef in db_trap (type=3, code=0) > at /usr/src/sys/ddb/db_main.c:270 > #5 0xffffffff80891d02 in kdb_trap (type=3, code=0, tf=0xfffffe00c62a1680) > at /usr/src/sys/kern/subr_kdb.c:727 > #6 0xffffffff80dd53c3 in trap (frame=0xfffffe00c62a1680) > at /usr/src/sys/amd64/amd64/trap.c:604 > #7 0xffffffff80dd6718 in trap_check (frame=0xfffffe00c62a1680) > at /usr/src/sys/amd64/amd64/trap.c:664 > #8 > #9 breakpoint () at /usr/src/sys/amd64/include/cpufunc.h:66 > #10 0xffffffff808910d0 in kdb_enter (why=0xffffffff80eaaf0b "panic", > msg=0xffffffff80eaaf0b "panic") at /usr/src/sys/kern/subr_kdb.c:505 > #11 0xffffffff8081dbfe in vpanic ( > fmt=0xffffffff80e80f73 "Duplicate free of %p from zone %p(%s) slab %p(%d)", ap=0xfffffe00c62a1850) at /usr/src/sys/kern/kern_shutdown.c:906 > #12 0xffffffff8081d6b0 in panic ( > fmt=0xffffffff80e80f73 "Duplicate free of %p from zone %p(%s) slab %p(%d)") > at /usr/src/sys/kern/kern_shutdown.c:843 > #13 0xffffffff80caaec5 in uma_dbg_free (zone=0xfffffe00dc9d9800, > slab=0xfffff80007ee0fd8, item=0xfffff80007ee0000) > at /usr/src/sys/vm/uma_core.c:5664 > #14 0xffffffff80c9faf5 in item_dtor (zone=0xfffffe00dc9d9800, > item=0xfffff80007ee0000, size=544, udata=0x0, skip=SKIP_NONE) > at /usr/src/sys/vm/uma_core.c:3418 > #15 0xffffffff80c9eec7 in uma_zfree_arg (zone=0xfffffe00dc9d9800, > item=0xfffff80007ee0000, udata=0x0) at /usr/src/sys/vm/uma_core.c:4374 > #16 0xffffffff802e5a89 in uma_zfree (zone=0xfffffe00dc9d9800, > item=0xfffff80007ee0000) at /usr/src/sys/vm/uma.h:404 > #17 0xffffffff802dcfa6 in xpt_free_ccb (free_ccb=0xfffff80007ee0000) > at /usr/src/sys/cam/cam_xpt.c:4674 > #18 0xffffffff802db639 in camperiphdone (periph=0xfffff8005d68bd00, > done_ccb=0xfffff80007797cc0) at /usr/src/sys/cam/cam_periph.c:1427 > #19 0xffffffff802e59b6 in xpt_done_process (ccb_h=0xfffff80007797cc0) > at /usr/src/sys/cam/cam_xpt.c:5491 > #20 0xffffffff802e811e in xpt_done_td (arg=0xffffffff81143c00 ) > at /usr/src/sys/cam/cam_xpt.c:5546 > #21 0xffffffff807ac0ea in fork_exit (callout=0xffffffff802e7f20 , > arg=0xffffffff81143c00 , frame=0xfffffe00c62a1c00) > at /usr/src/sys/kern/kern_fork.c:1083 > #22 > So, I did ``git reset --hard 8dc96b74edb844bb621afeba38fe4af104b13120'', which was the penultimate commit made by trasz to clear CCBs on the stack after he committed 3394d4239b85b5577845d9e6de4e97b18d3dba58, the change to allocate CCBs in UMA. Note that I only built the kernel and not world. I tried to reset to 3394d4239b85b5577845d9e6de4e97b18d3dba58 itself, but without the following commits for CCBs on the stack the kernel paniced during startup in AHCI. Anyway, this is the minimum set of changes relevant to the uma_ccbs story and also results in a panic identical to the one listed above when I set kern.cam.da.enable_uma_ccbs=1 and turn on the external USB disk. So, Warner is probably right and at least the da_uma_ccbs commits should be reverted until more research can be done on why the panic happens. The ada_uma_ccbs commits do not cause any problems in my experience and could probably be left in the kernel. -- Gary Jennejohn