From nobody Fri Nov 28 13:36:39 2025 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4dHvV94K15z6HSx4 for ; Fri, 28 Nov 2025 13:37:05 +0000 (UTC) (envelope-from rb@gid.co.uk) Received: from gid2.gid.co.uk (ns0.gid.co.uk [IPv6:2001:470:94de::240]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "gid2.gid.co.uk", Issuer "gid2.gid.co.uk" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4dHvV83CD7z3m3r for ; Fri, 28 Nov 2025 13:37:04 +0000 (UTC) (envelope-from rb@gid.co.uk) Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=pass (policy=none) header.from=gid.co.uk; spf=pass (mx1.freebsd.org: domain of rb@gid.co.uk designates 2001:470:94de::240 as permitted sender) smtp.mailfrom=rb@gid.co.uk Received: from mx0.gid.co.uk (mx0.gid.co.uk [194.32.164.250]) by gid2.gid.co.uk (8.15.2/8.15.2) with ESMTP id 5ASDau4r006200; Fri, 28 Nov 2025 13:36:56 GMT (envelope-from rb@gid.co.uk) Received: from smtpclient.apple ([89.248.30.154]) by mx0.gid.co.uk (8.14.2/8.14.2) with ESMTP id 5ASDap8A073953; Fri, 28 Nov 2025 13:36:51 GMT (envelope-from rb@gid.co.uk) Content-Type: text/plain; charset=utf-8 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3826.700.81.1.3\)) Subject: Re: any way to recover from I/O hang? From: Bob Bishop In-Reply-To: <458400081.20251126171621@yahoo.com> Date: Fri, 28 Nov 2025 13:36:39 +0000 Cc: "freebsd-hackers@freebsd.org" Content-Transfer-Encoding: quoted-printable Message-Id: <23A5EAA0-F4E3-4195-92B1-89A090775282@gid.co.uk> References: <458400081.20251126171621.ref@yahoo.com> <458400081.20251126171621@yahoo.com> To: Anthony Pankov X-Mailer: Apple Mail (2.3826.700.81.1.3) X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.30 / 15.00]; SUBJECT_ENDS_QUESTION(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; MV_CASE(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[gid.co.uk,none]; R_SPF_ALLOW(-0.20)[+mx]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; TO_DN_EQ_ADDR_SOME(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US]; FREEMAIL_TO(0.00)[yahoo.com]; MID_RHS_MATCH_FROM(0.00)[]; R_DKIM_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_LAST(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MLMMJ_DEST(0.00)[freebsd-hackers@FreeBSD.org]; ARC_NA(0.00)[] X-Rspamd-Queue-Id: 4dHvV83CD7z3m3r Hi, > On 26 Nov 2025, at 14:16, Anthony Pankov = wrote: >=20 >=20 > Recently I'm again facing a situation where some UFS (?) error on one = partition forced me to reboot whole server and interrupt people's work. > As I understand the brief is: >=20 > Reason: A process in state 'D' (uninterruptible wait/sleep) is blocked = on an I/O operation (e.g., waiting for a network file share, a disk = operation, or a device that is no longer responding) at the kernel = level. The process is not awaken to handle signals until the condition = it is waiting for is resolved. > Solution: You cannot kill a 'D' state process with any signal, = including SIGKILL (kill -9). The condition must be resolved (e.g., = fixing the network connection, waiting for the I/O to time out, or = potentially unmounting the filesystem with umount -f if it's a mounted = network share). In some rare cases involving buggy kernel code or = hardware failure, a reboot may be the only option. >=20 > In my case there was a jailed samba which share one ufs-formatted = partition. The samba processes hangs one by one and sharing has stopped = it work. > There was no message in log. Geom reported no errors. Disk's has no = error. It seems that there was some introduced inconsistency in UFS. > My thoughts was to stop/kill samba, remove samba's jail, unmount = partition and do fsck. >=20 > But 'jail -r' exit with something 'rc.shutdown exited ... 9' and left = jail running. pskill -KILL for samba processes say nothing and do = nothing. > 'umount -f' say 'Device is busy'. Various utilities such as 'df' = hangs. So I forced to shutdown the server which has other important and = workable service to resolve the situation. >=20 > I wander is there any way to treat such a cases? May be 'umount -f' = can have more power... FWIW I have found in the past that SIGBUS will sometimes unstick a = process that SIGKILL won=E2=80=99t shift. > P.S. Previous situation was when I do simple (as I think) experiments = with ggated and have forced to reboot server when 'mount' hung. >=20 > --=20 > Best regards, > Anthony Pankov mailto:anthony.pankov@yahoo.com >=20 >=20 -- Bob Bishop rb@gid.co.uk