From nobody Sat Dec 20 15:10:59 2025 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4dYSXk0cjmz6MHd6 for ; Sat, 20 Dec 2025 15:11:18 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-pj1-x102f.google.com (mail-pj1-x102f.google.com [IPv6:2607:f8b0:4864:20::102f]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4dYSXj10tDz3fWN for ; Sat, 20 Dec 2025 15:11:17 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-pj1-x102f.google.com with SMTP id 98e67ed59e1d1-34c27d14559so2182577a91.2 for ; Sat, 20 Dec 2025 07:11:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20230601.gappssmtp.com; s=20230601; t=1766243471; x=1766848271; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=UmqsTbOEu4dShyRO0wAycrm1hXR0UGaa8mBfF15i9zM=; b=JJG/jh2nUyhSkAZPKazJh02sKalf/+oIN15G3IrLXX2CyHk9YmQOEFFj+iEx7pUvsy iXhVB6rrYjZnZTExuUpTWApZ0Sz63xNdOWZFXvy24UfDxVKYDEL+oM+s4YPGdmlMv9EP kdR2C2I4R4hrV10XxuCc16CtPrBj9ozpMxhzTsj69gK5vTxzcaGVl80UnKKKvplHKmc+ QyGDEx9NEGotX+9OGFnb23V4i1oWybtHGW06SKevkth6BGAdGYBuhQnSRNEcC6+ayJg+ ozIQz9fGmYzuWxQE/efWkdSIk47RODbHb+/+9LaXxUIpHY5OGwUgI+6BncNC4cB7NS5Y bj5g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1766243471; x=1766848271; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=UmqsTbOEu4dShyRO0wAycrm1hXR0UGaa8mBfF15i9zM=; b=ieO7eQomtordv3nO0bInK9Y1cuAzEcV5EzF06Cw+pphR71550xS3AYIuF505hE7OHy BXTsFDRJkYPZra/dH/y9Y73R6U4b5iffEjDWmqShAjdDtV/yA4OdyJguJG7XikJAZinG VdgPHCZXmG1rlhPRdr+TLWXXUpeVNr4v26CtPU/w2yyDKbX2ES6N3f2J/tctAaXhoU6e g5gDl3gzIuw65rvRQQH8w1w9jBJYzDQtXABm6Y5FLbrPegz9SEuNg85ZA1sIEUzotgVv /7LOtf5z6/2CWT/rqGI2g3IMx4t1cpt66ZqFTDjU/QnZO4bQ8W4omCEOaVrPR57RaqEG 3jaQ== X-Gm-Message-State: AOJu0YybqIL04yKGPqKX7f3P9aDEqCb2P6Cl8VUkffyFL4PoaodkEBiN 6FfWcsk++Bl1o8oqT5M8qvExeah07hmpoLgV2uVo1LC01jFL51yDGIgfSdYwztO628Q5IZ7Jxtt De3ml1KBCRZoNKfkWNc7wD8JY/aj8zY+t4PHz2FNK9gC0cH0ifsT58RXwHg== X-Gm-Gg: AY/fxX5KHsQxGFLF/8TUryjhEYeAk/mF4L/E8/HPJio2v1tRsXZMBp7/DFkbpSQvfmd EjBas/Bn4YD1BPX8Uyh8jCPxU94paCnMI1ijON2oIWQYBPXqJP3JRpnI6jftYx7p5rsdCBSB01o SxHfaa76eF9Xcn30FS5lGgpiZXwqNjALDMl961H9j+VOHFT87EFVhKfn829GQF/h3Zq2AQuKIs/ YPnKJAho2YlI3UXmy6TW+RZ9RjMLkr8k0V4Y3CpAL/R499JajBhVNXz79v8us7h+QIWeUU= X-Google-Smtp-Source: AGHT+IEJm3Ip+M4aF5ladtA7GiUNPL/gRWDrPuRweDxdS9gNbxquOECcu/FRpDBiqH6snXNLDroNPFFa6PTNx29Pp+4= X-Received: by 2002:a17:90a:d890:b0:34e:5516:3fe3 with SMTP id 98e67ed59e1d1-34e92129ad6mr5169655a91.11.1766243470561; Sat, 20 Dec 2025 07:11:10 -0800 (PST) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 References: <20251220141124.1606aa7c@thor.sb211.local> In-Reply-To: <20251220141124.1606aa7c@thor.sb211.local> From: Warner Losh Date: Sat, 20 Dec 2025 08:10:59 -0700 X-Gm-Features: AQt7F2rQY4rUpZFBwQp8Rxb-8Br8Rn2eviQzb5mu42GLQ5VzsHlv1PEa9Eqze8M Message-ID: Subject: Re: CURRENT: havock: elf_load_section: truncated ELF file To: A FreeBSD User Cc: FreeBSD CURRENT Content-Type: multipart/alternative; boundary="000000000000a6622d0646639a86" X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Queue-Id: 4dYSXj10tDz3fWN --000000000000a6622d0646639a86 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Sat, Dec 20, 2025 at 6:12=E2=80=AFAM A FreeBSD User wrote: > Hello, > > recently a small server running recent CURRENT with a UFS basesd system > SSD (NVMe) and a data > graveyard based on RAID level 5 with ZFS (attached to a Fujitsu HBA > controler) gets corrupted > because of "loosing" a driver - this time the system reported TWO drives = a > removed froma RAID > level 5 - which is like a death sentence. > > I guess this is a fallout of the recently changed timie parameters to the > CAM infrastructure > (I can't find any notes on this in man cam, so I feel lost). > Unlikely, but you can set this in the boot loader: kern.cam.tur_timeout=3D60 kern.cam.inquiry_timeout=3D60 kern.cam.modesense_timeout=3D60 and see if that works. You should see new errors on boot if his is the issue. Can you share a dmesg? I kinda doubt they'd cause the issues that you've had. If disks are gone, then there'd be different errors to what you are seeing, I'd think. To recover, your best bet is to use a USB stick from one of the release or snapshots. Warner > A very desastrous side effect of this crash was the inability to reboot > the box (CURRENT pre- > 16.0-CURRENT #11 master-n282659-7f39d05b67ae: Sat Dec 20 09:35:32 CET > 2025amd64, the runtime > system was from 16th or 17th of December). > After several tenth of minutes I had to hadr reboot the box - with obviou= s > data loss on the > system SSD. And here my problems start to turn into a mess. > > After the first initial reboot I performed a fsck -fy, rebootet and > whitnessed that > jails didn't come up anymore and SSHD didn't work. So I installed prior t= o > the crash already > compiled CURRENT from /usr/src which is "master-n282659-7f39d05b67ae" (as > the sibling box which > is runnig great by the way, but different CPU and smaller RAID, but also > system SSD based on > UFS filesystem, same HBA. So CURRENT seem to operate in general on simila= r > hardware. > > After the second reboot with the old kernel the box in question went into > debugger, rebooting > in single user mode and performing fsck -fy revealed a lot of repairs on > the first partitions, > /, /var, /usr. After a reboot I realized that most services now are broke= n > - jails do not > start, sshd doesn't start and the whole system is going into multiuser, > but seems to have > serious problems. > > uname -a remains empty > cd /usr/src; make buildworld returns immediately empty, no further action > service ldconfig start also returns complete empty on console > > Several onboard/base tools simply return nothing. > > trying "/resucue/sh" (install date indicates 20th of December, so it is > the latest ) seems to > give me the first indication of something has terribly gone wrong or even > /rescue/vi (to edit > loader to change to boot.old): > > elf_load_section: truncated ELF file > Abort trap > > Checking /boot/kernel, /lib, /usr/lib, /bin or /sbin seems to be intakt > (as far as I can > check, all timestamps are 20th Dec 2025, 9:48 UTC). > > Well, since this is not the first time I ran into some problems using > CURRENT, the outage due > to two lost ZFS drives after the recent chenges seems worthy to make some > note here. > Can you provide error messages at boot for this? You talk about fsck and about ZFS, so I'm a little confused as to your setup. Warner > The other question would be how to fix: one strategy would be to boot fro= m > an official image > from flash drive and try to perform a "make installkernel installworld". > Maybe there is > another way idicativ to that what I described above ... > > Thanks in advance, > > oh > > > -- > > A FreeBSD user > --000000000000a6622d0646639a86 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Sat, Dec 20,= 2025 at 6:12=E2=80=AFAM A FreeBSD User <freebsd@walstatt-de.de> wrote:
Hello,

recently a small server running recent CURRENT with a UFS basesd system SSD= (NVMe) and a data
graveyard based on RAID level 5 with ZFS (attached to a Fujitsu HBA control= er) gets corrupted
because of "loosing" a driver - this time the system reported TWO= drives a removed froma RAID
level 5 - which is like a death sentence.

I guess this is a fallout of the recently changed timie parameters to the C= AM infrastructure
(I can't find any notes on this in man cam, so I feel lost).

Unlikely, but you can set this in the boot loader= :
kern.cam.tur_timeout=3D60
kern.cam.inquiry_timeout=3D= 60
kern.cam.modesense_timeout=3D60

and s= ee if that works.=C2=A0 You should see new errors on boot if his is the iss= ue. Can you share a dmesg?

I kinda doubt they'= d cause the issues that you've had. If disks are gone, then there'd= be different errors to what you are seeing, I'd think.=C2=A0

To recover, your best bet is to use a USB stick from one of= the release or snapshots.

Warner
=C2=A0=
A very desastrous side effect of this crash was the inability to reboot the= box (CURRENT pre-
16.0-CURRENT #11 master-n282659-7f39d05b67ae: Sat Dec 20 09:35:32 CET 2025a= md64, the runtime
system was from 16th or 17th of December).
After several tenth of minutes I had to hadr reboot the box - with obvious = data loss on the
system SSD. And here my problems start to turn into a mess.

After the first initial reboot I performed a fsck -fy, rebootet and whitnes= sed that
jails didn't come up anymore and SSHD didn't work. So I installed p= rior to the crash already
compiled CURRENT from /usr/src which is "master-n282659-7f39d05b67ae&q= uot; (as the sibling box which
is runnig great by the way, but different CPU and smaller RAID, but also sy= stem SSD based on
UFS filesystem, same HBA. So CURRENT seem to operate in general on similar = hardware.

After the second reboot with the old kernel the box in question went into d= ebugger, rebooting
in single user mode and performing fsck -fy revealed a lot of repairs on th= e first partitions,
/, /var, /usr. After a reboot I realized that most services now are broken = - jails do not
start, sshd doesn't start and the whole system is going into multiuser,= but seems to have
serious problems.

uname -a remains empty
cd /usr/src; make buildworld returns immediately empty, no further action <= br> service ldconfig start also returns complete empty on console

Several onboard/base tools simply return nothing.

trying "/resucue/sh" (install date indicates 20th of December, so= it is the latest ) seems to
give me the first indication of something has terribly gone wrong or even /= rescue/vi (to edit
loader to change to boot.old):

elf_load_section: truncated ELF file
Abort trap

Checking /boot/kernel, /lib, /usr/lib, /bin or /sbin seems to be intakt (as= far as I can
check, all timestamps are 20th Dec 2025, 9:48 UTC).

Well, since this is not the first time I ran into some problems using CURRE= NT, the outage due
to two lost ZFS drives after the recent chenges seems worthy to make some n= ote here.

Can you provide error message= s at boot for this? You talk about fsck and about ZFS, so I'm a little = confused as to your setup.

Warner
=C2=A0=
The other question would be how to fix: one strategy would be to boot from = an official image
from flash drive and try to perform a "make installkernel installworld= ". Maybe there is
another way idicativ to that what I described above ...


=C2=A0
Thanks in advance,

oh


--

A FreeBSD user
--000000000000a6622d0646639a86--