BIOS/MBR drive enumeration order, FreeBSD dev nodes, GELI ZFS root, and ZFS boot

freebsd-questions:


I have a desktop computer:

     Intel DQ67SW motherboard
     Intel Core i7-2600S processor
     Corsair CMV8GX3M2A1333C9 2 @ 4 GB memory module kit
     Intel 520 Series SSD 60 GB


EFI is disabled in CMOS Setup.


FreeBSD was installed onto the SSD with a USB flash drive containing:

     FreeBSD-12.3-RELEASE-amd64-memstick.img


As follows:

     FreeBSD - Welcome to FreeBSD        Boot Multi user
     FreeBSD Installer - Welcome Install
     Keymap Selection            Continue with default keymap
     Set Hostname                        f4.tracy.holgerdanske.com
     Distribution Selection              <unselect all options>
     Partitioning                        Auto (ZFS)
         Pool Type/Disks         stripe: 1 disk
                                     ada0 INTEL SSDSC2CW060A3
         Pool Name                       f4_zroot
         Force 4K Sectors                YES
         Encrypt Disks           YES
         Partition Scheme                MBR (BIOS)
         Swap Size                       2g
         Mirror Swap                     YES
         Encrypt Swap            YES
         >>> Install                     Proceed with installation
     ZFS Configuration
         Last Chance!            YES
         Enter a strong passphrase       ********
         Re-enter password               ********
     Root - New Password         ********
     Retype New Password         ********
     Network Configuration               em0
     Configure IPv4                      Yes
     Use DHCP                    Yes
     Configure IPv6                      No
     Search                              tracy.holgerdanske.com
     IPv4 DNS #1                 192.168.5.1
     IPv4 DNS #2                 208.67.222.222
     CMOS clock set to UTC               Yes
     Time Zone
         Region                  America -- North and South
         Country or region               United States of America
         Zone                    Pacific
     Time & Date - month, year, day      <verify>
     Time & Data - time          <verify>
     Services to start at boot   sshd, ntpdate, ntpd, powerd
     System hardening options    <none>
     Add users                   No
     Final configuration         Exit
     Open shell                  No
     Complete                    Reboot


On May 29, 2022, I upgraded to:

     2022-06-20 10:48:55 toor@f4 ~
     # freebsd-version ; uname -a
     12.3-RELEASE-p5
     FreeBSD f4.tracy.holgerdanske.com 12.3-RELEASE-p5 FreeBSD 
12.3-RELEASE-p5 GENERIC  amd64


When I boot the machine, I see a prompt:

     GELI Passphrase:


If I enter the correct passphrase, boot succeeds.


If I enter an incorrect passphrase, I see another prompt later in boot:

     Enter passphrase for ada0s1d:


Note that "ada0" corresponds to the device node name of the SSD when 
FreeBSD was installed.


If I enter the correct passphrase, boot continues and succeeds.


I would now like to use the computer to examine an EIDE HDD.


I have installed a HBA:

     Promise Technology Ultra 133 TX2


Connected the HDD to the HBA:

     Maxtor 6L200P0


And set the CMOS Setup boot order so that the SSD is before the HDD.


When I boot the machine I see the first prompt:

     GELI Passphrase:


No matter what passphrase I enter, I always see the second prompt:

     Enter passphrase for ada1s1d:


Note that "ada1" no longer corresponds to the device node name of the 
SSD when FreeBSD was installed.  (I suspect the HDD is "ada0".)


No matter what passphrase I enter at the second prompt, it is always 
rejected.  FreeBSD will not boot.


It appears that GELI and/or ZFS for the root partition is tied to the 
FreeBSD device node name "ada0s1d" (?), which is tied to the BIOS drive 
numbering.  When I insert the HBA, the BIOS drive numbering changes, the 
root partition device node changes to "ada1s1d", GELI and/or ZFS breaks, 
and boot fails (?).


CMOS Setup does not appear to provide a mechanism for influencing BIOS 
drive enumeration order.


The HBA does not appear to have a POST hotkey/ setup program.


I recall encountering this device node name issue when cloning FreeBSD 
installations between SSD's (device node names "adaN") and USB flash 
drives (device node names "daN"), and that the work-around was to boot 
FreeBSD installation media into a rescue shell, mount the boot 
filesystem, and delete zfs/zpool.cache (?).


I would prefer a work-around that involves entering configuration 
settings during FreeBSD boot.


Please advise.


TIA,

David