Re: Missing disk partition devices and GPT lables in /dev/
Date: Tue, 28 Nov 2023 19:55:30 UTC
On Tue, Nov 28, 2023 at 12:51 PM Miroslav Lachman <000.fbsd@quip.cz> wrote: > I have strange problem on one of my machines - the disk devices in /dev/ > are only shown as ada0 and ada1 but no partition devices, no devices in > /dev/gpt/ or /dev/gptid/, these directories are completely missing even > though there are GPT labels. > This is FreeBSD 13.2-RELEASE-p5 amd64 on HP Microserver Gen8. Disk are 2 > Seagate NAS 4TB, manually partitioned with gpart utility, zpool created > and then all the date migrated by zfs send and zfs receive from old 2TB > disks. Everything works, system is running but device nodes in /dev/ are > missing. I noticed this only when I want to create gmirrored swap from > ada0p4 and ada1p4 but got "No such file or directory." > > There are only diskid style devices in /dev/diskid/ > > gpart show -l shows correct GPT labels for partitions > > # gpart show -l > => 40 7814037088 diskid/DISK-Z3010G8V GPT (3.6T) > 40 1024 1 bootG8V (512K) > 1064 40960 2 efiG8V (20M) > 42024 83886080 3 diskG8Vsys (40G) > 83928104 20971520 4 swapG8V (10G) > 104899624 7707033600 5 diskG8Vtank0 (3.6T) > 7811933224 2103904 - free - (1.0G) > > => 40 7814037088 diskid/DISK-Z302YC90 GPT (3.6T) > 40 1024 1 bootC90 (512K) > 1064 40960 2 efiC90 (20M) > 42024 83886080 3 diskC90sys (40G) > 83928104 20971520 4 swapC90 (10G) > 104899624 7707033600 5 diskC90tank0 (3.6T) > 7811933224 2103904 - free - (1.0G) > > > But gpart show -p does not show ada0p1 ada0p2 ..ada0pN, it shows diskid > style entries > > # gpart show -p > => 40 7814037088 diskid/DISK-Z3010G8V GPT (3.6T) > 40 1024 diskid/DISK-Z3010G8Vp1 freebsd-boot (512K) > 1064 40960 diskid/DISK-Z3010G8Vp2 efi (20M) > 42024 83886080 diskid/DISK-Z3010G8Vp3 freebsd-zfs (40G) > 83928104 20971520 diskid/DISK-Z3010G8Vp4 freebsd-swap (10G) > 104899624 7707033600 diskid/DISK-Z3010G8Vp5 freebsd-zfs (3.6T) > 7811933224 2103904 - free - (1.0G) > > => 40 7814037088 diskid/DISK-Z302YC90 GPT (3.6T) > 40 1024 diskid/DISK-Z302YC90p1 freebsd-boot (512K) > 1064 40960 diskid/DISK-Z302YC90p2 efi (20M) > 42024 83886080 diskid/DISK-Z302YC90p3 freebsd-zfs (40G) > 83928104 20971520 diskid/DISK-Z302YC90p4 freebsd-swap (10G) > 104899624 7707033600 diskid/DISK-Z302YC90p5 freebsd-zfs (3.6T) > 7811933224 2103904 - free - (1.0G) > > > # gpart list | grep label > label: bootG8V > label: efiG8V > label: diskG8Vsys > label: swapG8V > label: diskG8Vtank0 > label: bootC90 > label: efiC90 > label: diskC90sys > label: swapC90 > label: diskC90tank0 > > > # ll /dev/gpt/ > ls: /dev/gpt/: No such file or directory > > > # ll /dev/ada* > crw-r----- 1 root operator 0x73 Nov 27 00:00 /dev/ada0 > crw-r----- 1 root operator 0x81 Nov 27 00:00 /dev/ada1 > > > Sysctl seems normal to me. > > # sysctl kern.geom.label > kern.geom.label.disk_ident.enable: 1 > kern.geom.label.gptid.enable: 1 > kern.geom.label.gpt.enable: 1 > kern.geom.label.ufs.enable: 1 > kern.geom.label.ufsid.enable: 1 > kern.geom.label.reiserfs.enable: 1 > kern.geom.label.ntfs.enable: 1 > kern.geom.label.msdosfs.enable: 1 > kern.geom.label.iso9660.enable: 1 > kern.geom.label.flashmap.enable: 1 > kern.geom.label.ext2fs.enable: 1 > kern.geom.label.debug: 0 > > > This is the only way I can access disk partitions > > # ll /dev/diskid/ > total 0 > crw-r----- 1 root operator 0x7f Nov 27 00:00 DISK-Z3010G8V > crw-r----- 1 root operator 0x99 Nov 27 00:00 DISK-Z3010G8Vp1 > crw-r----- 1 root operator 0x9b Nov 27 00:00 DISK-Z3010G8Vp2 > crw-r----- 1 root operator 0x9d Nov 27 00:00 DISK-Z3010G8Vp3 > crw-r----- 1 root operator 0x9f Nov 27 00:00 DISK-Z3010G8Vp4 > crw-r----- 1 root operator 0xa1 Nov 27 00:00 DISK-Z3010G8Vp5 > crw-r----- 1 root operator 0xad Nov 27 00:00 DISK-Z302YC90 > crw-r----- 1 root operator 0xc5 Nov 27 00:00 DISK-Z302YC90p1 > crw-r----- 1 root operator 0xc7 Nov 27 00:00 DISK-Z302YC90p2 > crw-r----- 1 root operator 0xc9 Nov 27 00:00 DISK-Z302YC90p3 > crw-r----- 1 root operator 0xcb Nov 27 00:00 DISK-Z302YC90p4 > crw-r----- 1 root operator 0xcd Nov 27 00:00 DISK-Z302YC90p5 > > > I created two pools (sys and tank1) with adaN devices when there were 4 > disks and the system was booted from USB flash drive with FreeBSD 13.2; > current ada0 was ada2 and current ada1 was ada3. > > These commands were used > > # zpool create -m none sys mirror ada2p3 ada3p3 > > # zpool create -m none tank1 mirror ada2p5 ada3p5 > > But now, when the system is booted of these disks, they are reported > only by diskids. > > # zpool list -v > NAME SIZE ALLOC FREE CKPOINT EXPANDSZ > FRAG CAP DEDUP HEALTH ALTROOT > sys 39.5G 15.9G 23.6G - - > 12% 40% 1.00x ONLINE - > mirror-0 39.5G 15.9G 23.6G - - > 12% 40.2% - ONLINE > diskid/DISK-Z3010G8Vp3 40G - - - - > - - - ONLINE > diskid/DISK-Z302YC90p3 40G - - - - > - - - ONLINE > tank1 3.58T 1.22T 2.36T - - > 0% 34% 1.00x ONLINE - > mirror-0 3.58T 1.22T 2.36T - - > 0% 34.0% - ONLINE > diskid/DISK-Z3010G8Vp5 3.59T - - - - > - - - ONLINE > diskid/DISK-Z302YC90p5 3.59T - - - - > - - - ONLINE > > > I did similar setup and migration many times and never seen this problem. > > What can be wrong with this system? > The problem is a geom design point. When you have multiple aliases that you can open, the others are removed from the filesystem view when one is opened. It looks like from the above that the mirror-0 was created with the diskid, so that's the device that zfs opens, leaving the ada2 devices removed. Warner