From nobody Thu Feb 03 13:24:01 2022 X-Original-To: bugs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id ED785193D47B for ; Thu, 3 Feb 2022 13:24:01 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4JqKBT4sdjz4YZG for ; Thu, 3 Feb 2022 13:24:01 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 87D0C18B77 for ; Thu, 3 Feb 2022 13:24:01 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 213DO1Qd051967 for ; Thu, 3 Feb 2022 13:24:01 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 213DO1qo051966 for bugs@FreeBSD.org; Thu, 3 Feb 2022 13:24:01 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: bugs@FreeBSD.org Subject: [Bug 261690] NFSv4 mount on Linux client hangs during complex access patterns (gcc bootstrapping on client) Date: Thu, 03 Feb 2022 13:24:01 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 13.0-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: bf@cebitec.uni-bielefeld.de X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform op_sys bug_status bug_severity priority component assigned_to reporter Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Bug reports List-Archive: https://lists.freebsd.org/archives/freebsd-bugs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-bugs@freebsd.org MIME-Version: 1.0 ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1643894641; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=VLO1EznYHyUeg2CNBuep7BtIl0OuyhNBrsD1PGMQ/NI=; b=Ko96TuZ5tZcrejbGymG2pVzR9YVmn8a0o6ay5GxLy1p3bRT9pKLF6ETSi4YFZ9qZuaAEhF FD5WahSCzYDGVWEEa8/jHXQQ6Tpci9d7tIapKOub+ASe7B2GtsTkc6G926tmgmDogp4Xwd pSKNXbACyYm+1sMOFAnhli/Y453hgKXK7Mf5JpXhbHzHqDvBqq+GEZa4pqz9/tx533BxNp 8j933mKF95v0jfg8mmXI3Lwd424MaPeATn8MGUHAxigmIHc6Xo0/DbulgpkbdQoxylDEpn 0RyNVpvbTw5HbUxb4yTk/i2PbZQ7gS/GZJu/m9x42sC4R9xyRqtjI4tOznBViw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1643894641; a=rsa-sha256; cv=none; b=vBIB28uuLdVC6Xsbaa/Fywu+K8oM0cZ7653ZSyu27NBySBZ9Axw2y8L8B/MZjvk52gxsoz jZ/28abCh4PzNE7UYAFOMtqB4kQG0WXHlLjnIuZ/x2t5ZVPBmzN9FzTmm2v0fHtDHXmD5r wZaPIav8c/6FmIoFZ2fuHyvTB1PDf1cOSo+zAK1Km+UqMu/88MQjJnKd9RCHz0vJL3Zje4 Rg/QNllZ/zOzZpOH8P4oILwPmb3OEs1LmwWt6vmy2czZJToHkPDUGw0+1V6o5S+SheGT7w hzELqEe/ztFSPC6tHEOWQXURUevYgJJQ1tktltV4ZBQiGuuAO65mhLZ8X8bSJA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D261690 Bug ID: 261690 Summary: NFSv4 mount on Linux client hangs during complex access patterns (gcc bootstrapping on client) Product: Base System Version: 13.0-RELEASE Hardware: amd64 OS: Any Status: New Severity: Affects Some People Priority: --- Component: kern Assignee: bugs@FreeBSD.org Reporter: bf@cebitec.uni-bielefeld.de A ZFS dataset mounted from a FreeBSD-13.0-p7 NFS server via NFSv4.1 on a Linux Ubuntu 20.04.3 client with 5.4.0-81-generic kernel is used to build gcc-11.2.0 from source (sources and object files on NFS). The build process will become stuck in kernel space at different points in time. The same setup with FreeBSD 11.4-RELEASE-p5 as server works flawlessly. *** How to reproduce: Server: FreeBSD 13.0-RELEASE-p7 *** vfs.nfs.enable_uidtostring=3D1 vfs.nfsd.enable_stringtouid=3D1 vfs.nfsd.server_min_nfsvers=3D3 *** rpcbind_enable=3D"YES" nfs_server_enable=3D"YES" nfsv4_server_enable=3D"YES" nfs_reserved_port_only=3D"YES" nfsuserd_flags=3D"-manage-gids -usertimeout 10 -usermax 2300 20" mountd_enable=3D"YES" mountd_flags=3D"-r" rpc_lockd_enable=3D"YES" rpc_statd_enable=3D"YES" Client: Ubuntu 20.04.3 LTS 5.4.0-81-generic ZFS dataset exported sec=3Dsys, mounted on Linux client via NFSv4.1 (NFSv4.0 shows the same behaviour). On Linux client, vanilla build of gcc-11.2.0, sources and build dir on the same NFS mount: cd /vol/perf/bsd13-3x7/bf # this is on the NFS mount tar xf gcc-11.2.0.tar.gz cd gcc-11.2.0 ./contrib/download_prerequisites cd .. mkdir obj cd obj ../gcc-11.2.0/configure --prefix=3D/vol/bsd13-3x7/bf/gcc \ --enable-languages=3Dc,c++,fortran,go --disable-multilib make -j20 *** After some time the compiler processes become stuck in kernel space: [Wed Feb 2 17:01:39 2022] cc1plus D 0 940194 940193 0x00004320 [Wed Feb 2 17:01:39 2022] Call Trace: [Wed Feb 2 17:01:39 2022] __schedule+0x2e3/0x740 [Wed Feb 2 17:01:39 2022] schedule+0x42/0xb0 [Wed Feb 2 17:01:39 2022] io_schedule+0x16/0x40 [Wed Feb 2 17:01:39 2022] wait_on_page_bit+0x11c/0x200 [Wed Feb 2 17:01:39 2022] ? file_fdatawait_range+0x30/0x30 [Wed Feb 2 17:01:39 2022] wait_on_page_writeback+0x43/0x90 [Wed Feb 2 17:01:39 2022] __filemap_fdatawait_range+0x98/0x100 [Wed Feb 2 17:01:39 2022] filemap_write_and_wait+0x60/0xa0 [Wed Feb 2 17:01:39 2022] nfs_wb_all+0x1f/0x130 [nfs] [Wed Feb 2 17:01:39 2022] nfs4_file_flush+0x73/0xa0 [nfsv4] [Wed Feb 2 17:01:39 2022] filp_close+0x37/0x70 [Wed Feb 2 17:01:39 2022] __close_fd+0x7d/0xa0 [Wed Feb 2 17:01:39 2022] __x64_sys_close+0x22/0x50 [Wed Feb 2 17:01:39 2022] do_syscall_64+0x57/0x190 [Wed Feb 2 17:01:39 2022] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [Wed Feb 2 17:01:39 2022] RIP: 0033:0x7f4c3121a4ab [Wed Feb 2 17:01:39 2022] Code: Bad RIP value. [...] [Wed Feb 2 17:02:30 2022] nfs: server allerbeeke not responding, still try= ing [Wed Feb 2 17:02:30 2022] nfs: server allerbeeke not responding, still try= ing *** What happens on the wire: Capture on the FreeBSD server with: tcpdump -i mce0 -w /var/tmp/fbsd-nfs4-server.pcap host bildhorst pcap file is available on our Nextcloud: https://docs.cebitec.uni-bielefeld.de/s/n5ZYmjnYd2fjQZ3 On the close system call the Linux client seems to flush the file to disk w= ith a series of SEQUENCE,PUTFH,WRITE,GETATTR compounds which the FreeBSD server simply ceases to reply to after some time (right at the end of the capture). Also notable, a few calls seem to get multiple replies, e.g. frame 1604448 = in that capture (also SEQUENCE,PUTFH,WRITE,GETATTR) got two replies with diffe= rent seq IDs in 1604458 and 1604459. --=20 You are receiving this mail because: You are the assignee for the bug.=