From nobody Sat Apr 29 19:27:50 2023 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q7zyd1WQTz489JD; Sat, 29 Apr 2023 19:27:53 +0000 (UTC) (envelope-from jah@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q7zyd14LMz3QZD; Sat, 29 Apr 2023 19:27:53 +0000 (UTC) (envelope-from jah@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1682796473; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=X2zKEPELg+0AqjoxalmfTECiiw5mxhXdyi1Z0BsYgwE=; b=pu5GTH4ZdHwqk2wgbH65vBvAj86JSnUR+kPg/g63aQflqbTLH4syMQ5fFJAMv2EJIif+e5 KrycnoL6cdBaWg5hf2khBDAyCxtOfifY2O2gJ4PK+pRVf87NNwOFpKmTV9Trgy81rvE+nS ql6HVsjx8nUna1S5YCSNocvShykMS48GL890+YIMXPBaIACvwX9zBZV4TGMCDNDKLZ4DCK rFmh3G+0/zZs7xlSjtumcF7ZjGRGOniiDH9lW5+Qghl2rJWjcLUXS3MMd3HaoCdTJSYSWF lzA6sSxHc0Fj/um9CDdEBvQcZCSv38o4j/VrAQ5OUujjxd2GaVd6D7uLjOH9kg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1682796473; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=X2zKEPELg+0AqjoxalmfTECiiw5mxhXdyi1Z0BsYgwE=; b=W+WN389Secwl8Pfcvi5tfPm0AgrxHPpM3GRbWaDqdDbT4j0cbjVxvf4fPQZVCFvenB7QDO QYsg1hIaY6B+U8LuRV2v9bQLjTTOfhNOOxuKE/sLJB7xxhztmNMvm48Juw/MZ1P+WqGyHA 48hRgZKkiflz77Zq/njC1Ye/VjZvI3KTuOTWVAAX7EXp3ymskOBbugPnCML843/nMehbOX nWWi715KZMMT5fE/IDvmN2zawGee115ypREtgikdnxX/FF+UgbHFYA7Cytzt/YCAsKfMqt M22Pq6NY9tTdc5g6NFL2Pu1EHh0FJOY0YeH58GYgQPfqpkCc329Q8VptDJkb6g== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1682796473; a=rsa-sha256; cv=none; b=gscXbGJlLBYOLSYuWARCj7gObNkrPono0MoTSfA1cvFbfId0pTEVaP8eLrito+M9fl8TD2 AobfXvCL612c6rjxri8nbfzle5GyNTuoEQSNFWmFyfX0u4oqYUYQnMk9X8mOXWIzPub869 mi8JfiuvfR3HGoeakAGwl1nM3+BAJhvkpwOrHh2b/YeBQUY/n7yOcPH5/c1bEz1W//pzcI g98Boxjg+p0w8fsQI20/tH0uF7en4nGerHFBJXKIi9tIyt0CkpWHDRWuaNKVg2nlVu9P9y ikDmFvU1W06Hp4r40pUt36QpcWdUpvYH57Rl4mWdE5W+lMB/wumGBAbfPlK3vQ== Received: from corona (047-232-115-243.res.spectrum.com [47.232.115.243]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: jah) by smtp.freebsd.org (Postfix) with ESMTPSA id 4Q7zyc4vPjzvW1; Sat, 29 Apr 2023 19:27:52 +0000 (UTC) (envelope-from jah@freebsd.org) Date: Sat, 29 Apr 2023 14:27:50 -0500 From: "Jason A. Harmening" To: Dimitry Andric Cc: src-committers@freebsd.org, dev-commits-src-all@freebsd.org, dev-commits-src-branches@freebsd.org Subject: Re: git: 060699e91369 - stable/13 - Merge llvm-project release/15.x llvmorg-15.0.7-0-g8dfdcc7b7bf6 Message-ID: References: <202304092135.339LZMeJ081640@gitrepo.freebsd.org> <76DD2CB9-986B-4349-8F46-3B7BF63EB315@FreeBSD.org> List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <76DD2CB9-986B-4349-8F46-3B7BF63EB315@FreeBSD.org> X-ThisMailContainsUnwantedMimeParts: N On Sat, Apr 29, 2023 at 08:49:28PM +0200, Dimitry Andric wrote: > On 29 Apr 2023, at 20:33, Jason A. Harmening wrote: > > > > On Sun, Apr 09, 2023 at 09:35:22PM +0000, Dimitry Andric wrote: > >> The branch stable/13 has been updated by dim: > >> > >> URL: https://cgit.FreeBSD.org/src/commit/?id=060699e9136975d51d3f726b9785bdbac9a62ba6 > >> > >> commit 060699e9136975d51d3f726b9785bdbac9a62ba6 > >> Author: Dimitry Andric > >> AuthorDate: 2023-01-14 16:33:24 +0000 > >> Commit: Dimitry Andric > >> CommitDate: 2023-04-09 14:54:52 +0000 > >> > >> Merge llvm-project release/15.x llvmorg-15.0.7-0-g8dfdcc7b7bf6 > >> > >> This updates llvm, clang, compiler-rt, libc++, libunwind, lld, lldb and > >> openmp to llvmorg-15.0.7-0-g8dfdcc7b7bf6. > >> > >> PR: 265425 > >> MFC after: 2 weeks > > > > This MFC of llvm15 appears to have completely broken the Intel IOMMU > > driver on my stable/13 machine. After this series of commits, any > > downstream DMA seems to produce an IOMMU translation fault, which > > renders the machine completely unusable: no nvme boot disk, no usb > > keyboard, etc. > > > > The faults I see look something like this: > > > > DMAR4: ahci0: pci0:17:5 sid 8d fault acc 0 adt 0x0 reason 0x3 addr 26000 > > > > It's a bit surprising to see a toolchain upgrade produce breakage like > > this, but that's what git bisect clearly tells me. I wonder if some of > > the IOMMU control structures might be defined as C bitfields and the new > > compiler is emitting them differently? Also, was any breakage like this > > observed when -current was upgraded to llvm15 several months ago? > > I haven't heard anything about such breakage, no. > > > > More generally, this is the second time in as many months I've had to > > deal with IOMMU breakage on -stable. I can't imagine I'm the only > > person who sees value in running with DMA remapping enabled; do we need > > a dedicated DMAR-enabled machine in the cluster to smoke-test changes > > like this? More generally, should we avoid MFCing high-risk changes > > like this? > > Since there were very few bug reports, it was not deemed high risk. > > In any case, it would be good to get the bottom of what is causing the > problem, so is there any way you can isolate which code seems to be > going "bad"? > > For example, if this problem affects code in sys/dev/iommu, is there > some way you can compile that part with -O1, or with an older version > of clang (from ports), to see if the problem goes away? I did try removing all custom make.conf settings (previously I just had CPUTYPE?=icelake-server), but that didn't change the behavior. Before I try further build tweaks, I'd like to ask if the IOMMU fault report can provide guidance here? AFAICT all the faults I'm getting show "reason 0x3". If I'm reading the VT-d spec correctly, FR=0x3 indicates an invalid context entry, in other words there's something the hardware doesn't like in the way the address width or pagetable base is configured for the PCIe requestor. > > -Dimitry >