Re: zpools no longer exist after boot

From: Ronald Klop <ronald-lists_at_klop.ws>
Date: Thu, 28 Nov 2024 13:58:29 UTC
Btw:

The /etc/rc.d/zpool script looks into these cachefiles:

for cachefile in /etc/zfs/zpool.cache /boot/zfs/zpool.cache; do

I didn’t check where the cachefile pool property is used. 

Hope this helps resolving the issue. Or maybe helps you to provide more information about your setup. 

Regards,
Ronald. 

Van: Ronald Klop <ronald-lists@klop.ws>
Datum: 28 november 2024 14:50
Aan: Dennis Clarke <dclarke@blastwave.org>
CC: Current FreeBSD <freebsd-current@freebsd.org>
Onderwerp: Re: zpools no longer exist after boot

> 
> 
> 
> Are the other disks available at the moment the boot process does zpool import?
> 
> Regards,
> Ronald
> 
> Van: Dennis Clarke 
> Datum: 28 november 2024 14:06
> Aan: Current FreeBSD 
> Onderwerp: zpools no longer exist after boot
> 
>> 
>> This is a baffling problem wherein two zpools no longer exist after
>> boot. This is :
>> 
>> titan# uname -apKU
>> FreeBSD titan 15.0-CURRENT FreeBSD 15.0-CURRENT #1 main-n273749-4b65481ac68a-dirty: Wed Nov 20 15:08:52 GMT 2024 root@titan:/usr/obj/usr/src/amd64.amd64/sys/GENERIC-NODEBUG amd64 amd64 1500027 1500027
>> titan#
>> 
>> titan# zpool list
>> NAME   SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP HEALTH  ALTROOT
>> t0     444G  91.2G   353G        -         -    27%    20%  1.00x ONLINE  -
>> titan#
>> 
>> The *only* zpool that seems to exist in any reliable way is the little
>> NVME based unit for booting. The other two zpools vanished and yet the
>> devices exist just fine :
>> 
>> titan#
>> titan# camcontrol devlist
>>         at scbus0 target 0 lun 0 (pass0,ada0)
>>         at scbus1 target 0 lun 0 (pass1,ada1)
>>    at scbus2 target 0 lun 0 (ses0,pass2)
>>    at scbus6 target 0 lun 0 (ses1,pass3)
>>   at scbus7 target 0 lun 1 (pass4,nda0)
>>              at scbus8 target 0 lun 0 (da0,pass5)
>> titan#
>> titan# nvmecontrol devlist
>>   nvme0: SAMSUNG MZVKW512HMJP-000L7
>>      nvme0ns1 (488386MB)
>> titan#
>> titan# zpool status t0
>>    pool: t0
>>   state: ONLINE
>> status: Some supported and requested features are not enabled on the pool.
>>          The pool can still be used, but some features are unavailable.
>> action: Enable all features using 'zpool upgrade'. Once this is done,
>>          the pool may no longer be accessible by software that does not support
>>          the features. See zpool-features(7) for details.
>>    scan: scrub repaired 0B in 00:00:44 with 0 errors on Wed Feb  7 09:56:40 2024
>> config:
>> 
>>          NAME        STATE     READ WRITE CKSUM
>>          t0          ONLINE       0     0     0
>>            nda0p3    ONLINE       0     0     0
>> 
>> errors: No known data errors
>> titan#
>> 
>> 
>> Initially I thought the problem was related to cachefile being empty for
>> these zpools. However if I set the cachefile to something reasonable
>> then the cachefile property vanishes at a reboot.  The file, of course, exists just fine :
>> 
>> titan# zpool get cachefile proteus
>> NAME     PROPERTY   VALUE      SOURCE
>> proteus  cachefile  -          default
>> titan#
>> titan# zpool set cachefile="/var/log/zpool_cache" proteus
>> titan# zpool get cachefile proteus
>> NAME     PROPERTY   VALUE                 SOURCE
>> proteus  cachefile  /var/log/zpool_cache  local
>> titan# ls -ladb /var/log/zpool_cache
>> -rw-r--r--  1 root wheel 1440 Nov 28 11:45 /var/log/zpool_cache
>> titan#
>> 
>> So there we have 1440 bytes of data in that file.
>> 
>> titan# zpool set cachefile="/var/log/zpool_cache" t0
>> titan# zpool get cachefile t0
>> NAME  PROPERTY   VALUE                 SOURCE
>> t0    cachefile  /var/log/zpool_cache  local
>> titan#
>> titan# ls -ladb /var/log/zpool_cache
>> -rw-r--r--  1 root wheel 2880 Nov 28 11:46 /var/log/zpool_cache
>> titan#
>> 
>> Now we have 2 * 1440 bytes = 2880 bytes of some zpool cache data.
>> 
>> titan# zpool set cachefile="/var/log/zpool_cache" leaf
>> titan# zpool get cachefile leaf
>> NAME  PROPERTY   VALUE                 SOURCE
>> leaf  cachefile  /var/log/zpool_cache  local
>> titan#
>> titan# zpool get cachefile t0
>> NAME  PROPERTY   VALUE                 SOURCE
>> t0    cachefile  /var/log/zpool_cache  local
>> titan#
>> titan# zpool get cachefile proteus
>> NAME     PROPERTY   VALUE                 SOURCE
>> proteus  cachefile  /var/log/zpool_cache  local
>> titan#
>> titan# reboot
>> 
>>  From here on ... the only zpool that exists after boot is the local
>> little NVME samsung unit.
>> 
>> So here I can import those pools and then see that the cachefile property has been wiped out :
>> 
>> titan#
>> titan# zpool import proteus
>> titan# zpool import leaf
>> titan#
>> titan# zpool list
>> NAME      SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP HEALTH  ALTROOT
>> leaf     18.2T   984K  18.2T        -         -     0%     0%  1.00x ONLINE  -
>> proteus  1.98T   361G  1.63T        -         -     1%    17%  1.00x ONLINE  -
>> t0        444G  91.2G   353G        -         -    27%    20%  1.00x ONLINE  -
>> titan#
>> titan# zpool get cachefile leaf
>> NAME  PROPERTY   VALUE      SOURCE
>> leaf  cachefile  -          default
>> titan#
>> titan# zpool get cachefile proteus
>> NAME     PROPERTY   VALUE      SOURCE
>> proteus  cachefile  -          default
>> titan#
>> titan# zpool get cachefile t0
>> NAME  PROPERTY   VALUE      SOURCE
>> t0    cachefile  -          default
>> titan#
>> titan# ls -l /var/log/zpool_cache
>> -rw-r--r--  1 root wheel 4960 Nov 28 11:52 /var/log/zpool_cache
>> titan#
>> 
>> The cachefile exists and seems to have grown in size.
>> 
>> However a reboot will once again provide nothing but the t0 pool.
>> 
>> Baffled.
>> 
>> Any thoughts would be welcome.
>> 
>> -- 
>> --
>> Dennis Clarke
>> RISC-V/SPARC/PPC/ARM/CISC
>> UNIX and Linux spoken
>> 
>> 
>> 
>> 
>> 
> 
> 
> 
> 
> 
>