Re: Replacing a REMOVED drive in DEGRADED zpool
- Reply: Robert : "Re: Replacing a REMOVED drive in DEGRADED zpool"
- In reply to: Robert : "Re: Replacing a REMOVED drive in DEGRADED zpool"
- Go to: [ bottom of page ] [ top of archives ] [ this month ]
Date: Fri, 22 Aug 2025 06:04:11 UTC
On 8/21/25 21:43, Robert wrote: > On 8/21/2025 3:04 PM, David Christensen wrote: >> Please run and post the following commands to check if ada[0-3]p1 >> bootloader stages are the same: >> >> # cmp /dev/ada1p1 /dev/ada0p1 > Not until I resolve ada0. >> # cmp /dev/ada1p1 /dev/ada2p1 > root@db1:~ # cmp /dev/ada1p1 /dev/ada2p1 > /dev/ada1p1 /dev/ada2p1 differ: char 159233, line 417 >> # cmp /dev/ada1p1 /dev/ada3p1 > > Interesting? > > root@db1:~ # cmp /dev/ada1p1 /dev/ada3p1 > root@db1:~ # > Hmmm... freebsd-boot matches on ada1 and ada3, but not ada2 (?). That is strange. I wonder if ada2p1 is corrupt? I would try to find the matching bootloader file in /boot and compare it against all of ada[0-3]p1. See my post that crossed yours for commands. >> Please run and post the following commands to check if ada[0-3] MBR's >> are the same: >> >> # cmp <(dd if=/dev/ada1 count=1 status=none) <(dd if=/dev/ada0 count=1 >> status=none) >> >> # cmp <(dd if=/dev/ada1 count=1 status=none) <(dd if=/dev/ada2 count=1 >> status=none) >> >> # cmp <(dd if=/dev/ada1 count=1 status=none) <(dd if=/dev/ada3 count=1 >> status=none) > root@db1:~ # cmp <(dd if=/dev/ada1 count=1 status=none) <(dd if=/dev/ > ada0 count=1 status=none) > Missing name for redirect. > root@db1:~ # cmp <(dd if=/dev/ada1 count=1 status=none) <(dd if=/dev/ > ada2 count=1 status=none) > Missing name for redirect. > root@db1:~ # cmp <(dd if=/dev/ada1 count=1 status=none) <(dd if=/dev/ > ada3 count=1 status=none) > Missing name for redirect. I think we are using different shells, and yours does not support process substitution (?): 2025-08-21 22:58:13 dpchrist@laalaa ~ $ ssh root@f5 root@f5:~ # echo $SHELL /bin/csh root@f5:~ # exit logout Connection to f5 closed. 2025-08-21 22:58:37 dpchrist@laalaa ~ $ ssh toor@f5 2025-08-21 22:58:41 toor@f5 ~ # echo $SHELL /usr/local/bin/bash 2025-08-21 22:58:43 toor@f5 ~ # exit logout Connection to f5 closed. >> Please run and post the following commands to confirm that gmirror and >> gstripe are not in use. Do not bother to load if prompted: >> >> # gmirror status >> >> # gstripe status > > Yes, gmirror is in use, I'm guessing a `gmirror insert ada0p2` is neeed? > > root@db1:~ # gmirror status > Name Status Components > mirror/swap DEGRADED ada2p2 (ACTIVE) > ada3p2 (ACTIVE) > ada1p2 (ACTIVE) > root@db1:~ # gstripe status > gstripe: Command 'status' not available; try 'load' first. > So, the freebsd-swap partitions are mirrored and the freebsd-boot partitions are not are not. That is key information. You will need to make sure that you match the installed configuration when you configure the replacement disk. >> Please run and post the following command to confirm swap: >> >> # swapinfo > root@db1:~ # swapinfo > Device 1K-blocks Used Avail Capacity > /dev/mirror/swap 8388604 30216 8358388 0% > >> Please post (redact as necessary) /boot/loader.conf, /boot/ >> loader.conf.d/*, /etc/rc.conf, and /etc/fstab in case there is >> anything else we need to consider. > root@db1:~ # cat /boot/loader.conf > geom_mirror_load="YES" > kern.geom.label.disk_ident.enable="0" > kern.geom.label.gptid.enable="0" > cryptodev_load="YES" > zfs_load="YES" Looks reasonable. > root@db1:~ # ls -lah /boot/loader.conf.d/ > total 9 > drwxr-xr-x 2 root wheel 2B May 12 2022 . > drwxr-xr-x 15 root wheel 71B Jul 4 11:33 .. No configuration add-on files. > root@db1:~ # cat /etc/rc.conf > hostname="db1.REDACTED" > ifconfig_em0="inet REDACTED netmask 255.255.255.192" > defaultrouter="REDACTED.1" > > # Set dumpdev to "AUTO" to enable crash dumps, "NO" to disable > dumpdev="AUTO" > nrpe3_enable="YES" > sshd_enable="YES" > ntpdate_enable="YES" > ntpd_enable="YES" > named_enable="YES" > postgresql_enable="YES" > nfs_client_enable="YES" > nfs_server_enable="YES" > nfs_server_flags="-u -t -n 4" > rpcbind_enable="YES" > mountd_flags="-r" > mountd_enable="YES" > apache24_enable="YES" > postfix_enable="YES" > sendmail_enable="NONE" > sendmail_submit_enable="NO" > sendmail_outbound_enable="NO" > sendmail_msp_queue_enable="NO" > slapd_enable="YES" > slapd_flags='-h "ldapi://%2fvar%2frun%2fopenldap%2fldapi/ > ldap://0.0.0.0:389/ ldaps://0.0.0.0:636/"' > slapd_sockets="/var/run/openldap/ldapi" > mrtg_daemon_enable="YES" > saslauthd_enable="YES" > saslauthd_flags="-a ldap" That server has some complexity. > root@db1:~ # cat /etc/fstab > # Device Mountpoint FStype Options Dump Pass# > /dev/mirror/swap none swap sw 0 0 > REDACTED:/mnt/REDACTED /nfs/backup nfs rw 0 0 Regarding read-write /nfs/backup, I will assume that you have taken security precautions to protect backups and the backup server. >> Running and posting the commands above should help determine what to >> do with ada0p1 and adap2. > Thanks for the help. >>> Ok, I have a matching drive, so I don't plan on reusing the REMOVED >>> drive. I usually take the REMOVED one and examine it to see if it can >>> be re-inserted safely as needed. I was hoping the gpart backup/ >>> restore would be the equivalent of sgdisk in Linux that I've used >>> many times to duplicate a disk used for replacement in software RAID. >> >> >> I also use Linux, and FreeBSD gpart(8) backup and restore surprised me >> in a bad way. Please see the thread branch starting here: >> >> https://lists.freebsd.org/archives/freebsd-questions/2025- >> August/006883.html > Yeah, I read. So, gpart backup/restore can be used as long as I throw > the -l switch? >>>> I would use zpool-attach(8) to add the replacement ada0p3 as a >>>> mirror of ada1p3. >>> >>> Attach in place of zpool-replace? >> >> >> AIUI RTFM zpool(8) if you detach one of two drives in a mirror, the >> detached drive is forgotten, the mirror goes away, and the remaining >> disk becomes striped at the top level. When you install a new drive >> and want to create a mirror with a singular striped drive, use >> zpool(8) attach. > > This is the plan now thanks to all ya'lls help, unless I get away with > re-seating all and zpool online the disk. > > -- > Robert You are making good progress! :-) David