From nobody Thu Mar 24 15:26:28 2022 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 9825E1A3CE9D; Thu, 24 Mar 2022 15:26:28 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4KPTb83qw9z4ZbQ; Thu, 24 Mar 2022 15:26:28 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1648135588; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=fsMvqdo7s1gr3Z5C+m209Qg8hNBibojFMiF7MwewLAw=; b=iezCumRYdYhh413O073gBJVOBMG9Pw6hKC6EroUR8qR2BIKVNM89kSrCmO4Ck8r37f+oxq BMcvFyoNcnZE8MZRnIMNoGvYcg92KQuvr2To/pZRPfc8frCpdeQufOrojKoHjW8/rXSYB+ GIbHT4kQnG2IBfdIlwhBhjEk7nMaZ5OTgwX9OJjxR6aljBSeabmyb2tIW/sLPGUrZbBkT/ o3lNp++Zf/CHtiGGQRO5RMA+xxBnugCVZpK+hierIWGStgZJaGEUxsGw55fnqTCTxWyac6 lMbokrJzQNpaoEr5fehXsVmqnOSFw1Q+4L1RShbYnf3ESPnK+V4xSQv8jij5mA== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 59B6C1A811; Thu, 24 Mar 2022 15:26:28 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 22OFQSjv009340; Thu, 24 Mar 2022 15:26:28 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 22OFQSfr009339; Thu, 24 Mar 2022 15:26:28 GMT (envelope-from git) Date: Thu, 24 Mar 2022 15:26:28 GMT Message-Id: <202203241526.22OFQSfr009339@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: "Bjoern A. Zeeb" Subject: git: 246c39814567 - main - bhyve: Do not remove guest physical addresses from IOMMU host domain List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-main@freebsd.org X-BeenThere: dev-commits-src-main@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: bz X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 246c398145674e4a9337fd933a6e6da7f160118e Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1648135588; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=fsMvqdo7s1gr3Z5C+m209Qg8hNBibojFMiF7MwewLAw=; b=iiG0WigZHE/cxVzZfY2EhDubtLcLyL0wkNic9Gyy5pC8OSilWRv/rGehEgu1EXoGGy88Ut mp6oIiCbjAjYG8VHOkcA0VTajbSUutZz+J7phbGsaSzthOxDFIHAqW0rKv1LlY23GS0+kY lckIeVqtDwOJ4MkkMRXIh+zL11x4I+k8NEEVV0EnxmyUACBpGjHfVEOPfONxPjU8CsF9in ij8d1ApFKJF7DLNGvsMcZkULIHV1jlS17zBU1n59+9PDDUfN5u+sDF+u3Bu5i24azYZ5e1 0JuV3z1jVixGU9CF5I70c+r1Sgj4Z8l4GIb2r5PpvLU60G1hE0v0BavWEhW7ZQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1648135588; a=rsa-sha256; cv=none; b=mMNP9jlfWsaRYKeB9mMngwZhsz5tpU/ahH7WptygTmx4z2ZqJGzvrDd6ym6vMTJ9/GtIO2 522Bx3Kx6wFNVUm/s1z78bH9YnEWqMMfNLTtQad40lfpoatg0BE40VyueRW3YE4HRSLi/r DTelYIjhcsUqk2afMzBneJgMQuBo7TURI3Pai/schfpeCplm/Xo3k32gz9KdnmS3C+WA1h sIVv6nbnSamDqxdrNvGVLJCGakErCePg2rg+kr1tApfpxvtRT/W6LUFcZNm7RPo9GNXpbk ahdPSy1K9Yvg+dqP/9hmBrKqaJ/dNnO6hFwn5ppIY/aCh0vwNF8GxkZthHHI/Q== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by bz: URL: https://cgit.FreeBSD.org/src/commit/?id=246c398145674e4a9337fd933a6e6da7f160118e commit 246c398145674e4a9337fd933a6e6da7f160118e Author: Bjoern A. Zeeb AuthorDate: 2022-03-18 20:39:06 +0000 Commit: Bjoern A. Zeeb CommitDate: 2022-03-24 15:21:24 +0000 bhyve: Do not remove guest physical addresses from IOMMU host domain This permits I/O devices on the host to directly access wired memory dedicated to guests using passthru devices. Note that wired memory belonging to guests that do not use passthru devices has always been accessible by I/O devices on the host. bhyve maps guest physical addresses into the user address space of the bhyve process by mmap'ing /dev/vmm/. Device models pass pointers derived from this mapping directly to system calls such as preadv() to minimize copies when emulating DMA. If the backing store for a device model is a raw host device (e.g. when exporting a raw disk device such as /dev/ada as a drive in the guest), the host device driver (e.g. ahci for /dev/ada) can itself use DMA on the host directly to the guest's memory. However, if the guest's memory is not present in the host IOMMU domain, these DMA requests by the host device will fail without raising an error visible to the host device driver or to the guest resulting in non-working I/O in the guest. It is unclear why guest addresses were removed from the IOMMU host domain initially, especially only for VM's with a passthru device as the host IOMMU domain does not affect the permissions of passthru devices, only devices on the host. A considered alternative was using bounce buffers instead (D34535 is a proof of concept), but that adds additional overhead for unclear benefit. This solves a long-standing problem when using passthru devices and physical disks in the same VM. Thanks to: grehan (patience and help) Thanks to: jhb (for improving the commit message) PR: 260178 Reviewed by: grehan, jhb MFC after: 3 days Differential Revision: https://reviews.freebsd.org/D34607 --- sys/amd64/vmm/vmm.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/sys/amd64/vmm/vmm.c b/sys/amd64/vmm/vmm.c index faa7733581d1..45125cb92a7e 100644 --- a/sys/amd64/vmm/vmm.c +++ b/sys/amd64/vmm/vmm.c @@ -938,10 +938,8 @@ vm_iommu_modify(struct vm *vm, bool map) hpa = DMAP_TO_PHYS((uintptr_t)vp); if (map) { iommu_create_mapping(vm->iommu, gpa, hpa, sz); - iommu_remove_mapping(host_domain, hpa, sz); } else { iommu_remove_mapping(vm->iommu, gpa, sz); - iommu_create_mapping(host_domain, hpa, hpa, sz); } gpa += PAGE_SIZE;