From nobody Wed Mar 30 15:50:30 2022 X-Original-To: virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 380C11A561A5 for ; Wed, 30 Mar 2022 15:50:31 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4KT9r62prNz3kyj for ; Wed, 30 Mar 2022 15:50:30 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 2F3CB156EE for ; Wed, 30 Mar 2022 15:50:30 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 22UFoU2b045634 for ; Wed, 30 Mar 2022 15:50:30 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from bugzilla@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 22UFoUUI045633 for virtualization@FreeBSD.org; Wed, 30 Mar 2022 15:50:30 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: bugzilla set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 215740] [bhyve] utilizing passthru breaks raw device usage with virtio-blk | ahci-hd Date: Wed, 30 Mar 2022 15:50:30 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: misc X-Bugzilla-Version: 11.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: commit-hook@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1648655430; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=veMFJo70fQCq5QifGaGUOqN2MOWcLvRRGj3kJnm05xI=; b=jeCaDgG5ISwahtwofm8ZjDl4DjFmfJbuRFN8vot21zhNtz79VM5xUw1DzOsZSvdRiD/I9S dxCgX1sMY69eMwI6yEojpir+phWE4xa/C8vKRiT1+ENkT44U/15mkWUxwYuV4p/6wa3kpG Kp279njwBklpqE+1b9i99SpzLFIifiwWbPkMSUyAVt/mY6noMoKNp1G4OsrAd0UHlaSWYS qPhjTE1EjH9K8KSPuXyILtLVcgSlhGFy5iekJUdEQ84IB0KAVeXigrgD+vJIRZHGbYGrf9 memr3RqbJh7lLbjmpyHfiPd0UHFq0cQLPSuM/x9KhjvDSpEOEGx7nyn8wXDSpg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1648655430; a=rsa-sha256; cv=none; b=ydVeXq3ep50k/8579fSjMzvQP6bshSC6nqq0NvuQKOTtAXRDF3pxVqu4vK5JBf5juoKJlD 4C6WbYO1im8dnziI+OSeu8v+pf1Xj6dO0qN5BBmNasW0ANIxjzBR8fwl87cO/UwuQgy11E Le+Rh9lzK/ox8e9VrzYsHEdqDMlg442CFaG/9Wv9v8b3tLxY2UXPZQd/Aq4SAOFtJqBJjB P8CzjbsvGN+lk1PSWgLWKTwbzuP6tpyKe8Z23wGXNACGmgfQtbbTLJjmyVgmP9LPGYVcsF zjyiohA2t6oNWOBCqrlzvE/pwuH8NxTi4FYk4GlhBkzJ+ISl9DZ43pdbJMeVgw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D215740 --- Comment #10 from commit-hook@FreeBSD.org --- A commit in branch releng/13.1 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=3D1c6abf864ecd3bbf07ace2018f9aab45b= 6406ce2 commit 1c6abf864ecd3bbf07ace2018f9aab45b6406ce2 Author: Bjoern A. Zeeb AuthorDate: 2022-03-18 20:39:06 +0000 Commit: Bjoern A. Zeeb CommitDate: 2022-03-30 15:33:47 +0000 bhyve: Do not remove guest physical addresses from IOMMU host domain This permits I/O devices on the host to directly access wired memory dedicated to guests using passthru devices. Note that wired memory belonging to guests that do not use passthru devices has always been accessible by I/O devices on the host. bhyve maps guest physical addresses into the user address space of the bhyve process by mmap'ing /dev/vmm/. Device models pass pointers derived from this mapping directly to system calls such as preadv() to minimize copies when emulating DMA. If the backing store for a device model is a raw host device (e.g. when exporting a raw disk device such as /dev/ada as a drive in the guest), the host device driver (e.g. ahci for /dev/ada) can itself use DMA on the host directly to the guest's memory. However, if the guest's memory is not present in the host IOMMU domain, these DMA requests by the host device will fail without raising an error visible to the host device driver or to the guest resulting in non-working I/O in the guest. It is unclear why guest addresses were removed from the IOMMU host doma= in initially, especially only for VM's with a passthru device as the host IOMMU domain does not affect the permissions of passthru devices, only devices on the host. A considered alternative was using bounce buffers instead (D34535 is a proof of concept), but that adds additional overhead for unclear benefit. This solves a long-standing problem when using passthru devices and physical disks in the same VM. Approved by: re (gjb) Thanks to: grehan (patience and help) Thanks to: jhb (for improving the commit message) PR: 260178, 215740 Reviewed by: grehan, jhb Differential Revision: https://reviews.freebsd.org/D34607 (cherry picked from commit 246c398145674e4a9337fd933a6e6da7f160118e) (cherry picked from commit dd113f67dfb5bdaf5d8b3a87bb19924ad447494c) sys/amd64/vmm/vmm.c | 2 -- 1 file changed, 2 deletions(-) --=20 You are receiving this mail because: You are the assignee for the bug.=