Installing FreeBSD from USB flash drive - some experiments

[Clifton Royston]

  It seems like questions about installing FreeBSD from a USB drive
come up at regular intervals.  With the low price of flash drives these days, it would
be sort of nice to be able to offer a way for users to conveniently
roll their own USB installer images, or even to offer them for download
as well as the ISOs.

  I spent some time working on it this weekend, initially because I was
trying to get the current pfSense 1.2.2 (based on FreeBSD 7.0) to
install on the Geode-based fit-PC box.  (I eventually got there with
pfSense 1.2.3, based on 7.1.)

  While troubleshooting problems with installing pfSense from USB, I
ended up also building USB images from the 7.1-RELEASE bootonly and the
7.1-RELEASE disc1 ISOs.  To do this I've been hacking on Dario Freni's
fbsd-installiso2img.sh script from 2006, to convert FreeBSD or
FreeBSD-based ISOs to image files.  Initially I thought everything was
working fine, because the bootonly image came right up into sysinstall,
but when I dug a little deeper and tried with a USB image built from
the disc1 liveCD/install ISO, I found that it actually seems not to be
workable as it stands.

  I think I have found some limitations of the current sysinstall which
currently make it currently difficult to convert a working FreeBSD ISO
to an installable or live-disk flash image.   Booting the kernel is
easy.  Getting the rest of the way is harder.

  Background: AFAICT the release ISOs don't currently mount the media
they were booted from (da0a in this case.) Instead they load
boot/mfsroot.gz (compressed memory image) as the root fs, which is fine
and makes good sense, but they don't then mount the boot media anywhere
under it.  (My guess is that this was probably originally done to
support the floppy boot case.)

  So how can you direct it to install from the USB media?

  1) If you try to select media via the menu options, you can't select
USB specifically, nor specify an arbitrary device as the source
(/dev/da0a in this case), so you can't get the booted medium mounted to
install from, so you can't get the distributions accessible, so install
doesn't work.  Picking CD as the media seems to mean specifically an
cd or acd device.

  3) "Aha!" you say, "No problem.  If you're using a USB system, you
can instead choose install from filesystem." Good point, but the file
system on the USB flash is not mounted, and you can't mount it via the
menu.

  4) "So, the LiveCD, then?" The LiveCD function in sysinstall seems to
also currently require the install/liveCD image to be on a cd or acd
device.  With no such device, it won't mount anything and remains stuck
at the "Please insert a LiveCD or DVD" prompt until you cancel.

  5) "The emergency holographic shell, then?"  Yes, you can get into
the emergency holographic shell, with the very limited set of commands
from the mfsroot bin/sbin/stand.  However, if you try using the
emergency holographic shell to mount da0a - as far as I can tell, you
can't do so, because there is no mount in mfsboot's stand, only
mount_nfs.  (I'm guessing the "mount" operation for acd0 or cd0 is
coded into sysinstall?)  The usual mount command of course exists on the
media, but it's on the LiveCD portion which isn't mounted at this
point...

  As far as I could tell, from the state you're in after the system
boots, it's not possible to bootstrap up to where you can access the
contents of the flash drive.  Perhaps I'm just not creative enough.

  I'm looking for the simplest way to make this workable, so that one
could automatically generate a bootable flash drive image from a given
FreeBSD ISO.  Would adding an /etc/fstab mount entry for da0a (inside a
modified version of mfsroot.gz) be honored after loading the root, to
get the install medium premounted for bootstrapping?

  I'm attaching my hacked-up fbsd-installiso2img.sh in case anyone else
is interested in playing with it further.

  -- Clifton
----- cut here ----
#!/bin/sh
# fbsd-install-iso2img.sh
# Original version by Dario Freni 9/2006
# Enhancements by Clifton Royston 3/2009.
# License: Beerware

# You can set some variables here. Edit them to fit your needs.

# Set serial variable to 0 if you don't want serial console at all,
# 1 if you want comconsole and 2 if you want comconsole and vidconsole
serial=0

# Set nofstab=1 to not create any initial fstab on the USB drive;
# this makes the next two settings largely irrelevant.
nofstab=0

# Set rootperm=rw for root fs to mount r/w from the USB drive
#  (Should be unnecessary.)
rootperm=ro

# Set USBLABEL here or with -L label to label the image file system, 
#  to help the loader find the root file system when booting; 
#  otherwise the USB must come up as da0 to finish loading successfully.
USBLABEL=
lbparams=

# Set dopause=1 here or with -p to pause and allow review or editing of
#  the flash image before finalizing the image.
dopause=0

pause() {
 echo "Press enter to continue"
 read foo
}


set -u

if [ $# -ge 3 ]; then
  flag=$1
  if [ ${flag} = "-p" ]; then
    dopause=1
    shift
    flag=$1
  fi
  if [ ${flag} = "-n" ]; then
    nofstab=1
    shift
    flag=$1
  fi
  if [ ${flag} = "-L" ]; then
    shift;
    USBLABEL=$1; shift
    lbparams="-L ${USBLABEL}"
  fi
fi
if [ $# -lt 2 ]; then
  echo "Usage: $0 [-p] [-n] [-L vollabel] source-iso-path output-img-path"
  echo "	-p	pause for review before finalizing image"
  echo "	-n	don't update the /etc/fstab within the image"
  echo "	-L	set file system label on image, to help loader find it"
  exit 1
fi

isoimage=$1; shift
imgoutfile=$1; shift

export tmpdir=$(mktemp -d -t fbsdmount)

# Temp file and directory to be used later
export tmpfile=$(mktemp -t bsdmount)

export isodev=$(mdconfig -a -t vnode -f ${isoimage})

echo "#### Building bootable UFS image ####"

ISOSIZE=$(du -k ${isoimage} | awk '{print $1}')
SECTS=$((($ISOSIZE + ($ISOSIZE/5))*2))

# Root partition size

echo "Initializing image..."
dd if=/dev/zero of=${imgoutfile} count=${SECTS}
ls -l ${imgoutfile}
export imgdev=$(mdconfig -a -t vnode -f ${imgoutfile})

bsdlabel -w -B ${imgdev}
newfs -O1 ${lbparams} /dev/${imgdev}a

mkdir -p ${tmpdir}/iso ${tmpdir}/img

mount -r -t cd9660 /dev/${isodev} ${tmpdir}/iso
mount /dev/${imgdev}a ${tmpdir}/img

echo "Copying files to the image via cpio"
( cd ${tmpdir}/iso && find . -print -depth | cpio -dump ${tmpdir}/img )
# Dump doesn't work from an ISO file system, too bad.
# echo "Copying files to the image via dump/restore..."
## dump -0f - /dev/${isodev} | (cd ${tmpdir}/img && restore -r -f - ) 

#bzcat ${tmpdir}/iso/dist/root.dist.bz2 | mtree -PUr -p ${tmpdir}/img 2>&1 > /dev/null

if [ ${nofstab} -ne 1 ]; then
  echo "Saving original /etc/fstab as /etc/fstab.orig"
  mv ${tmpdir}/img/etc/fstab  ${tmpdir}/img/etc/fstab.orig
  echo "Replacing /etc/fstab, so loader can find root filesystem on flash!"
  if [ "${USBLABEL}" != "" ]; then
    echo "/dev/ufs/${USBLABEL} / ufs ${rootperm} 0 0" > ${tmpdir}/img/etc/fstab
    ## echo "devfs /dev devfs rw 0 0" >> ${tmpdir}/img/etc/fstab
  else 
    echo "/dev/da0a / ufs ${rootperm} 0 0" > ${tmpdir}/img/etc/fstab
    ## echo "devfs /dev devfs rw 0 0" >> ${tmpdir}/img/etc/fstab
  fi
else
  echo "Skipping write of image /etc/fstab"

fi

if [ ${serial} -eq 2 ]; then
        mv ${tmpdir}/img/boot.config ${tmpdir}/img/boot.config.orig
        mv ${tmpdir}/img/boot/loader.conf ${tmpdir}/img/boot/loader.conf.orig
        echo "-D" > ${tmpdir}/img/boot.config
        echo 'console="comconsole, vidconsole"' >> ${tmpdir}/img/boot/loader.conf
elif [ ${serial} -eq 1 ]; then
        mv ${tmpdir}/img/boot.config ${tmpdir}/img/boot.config.orig
        mv ${tmpdir}/img/boot/loader.conf ${tmpdir}/img/boot/loader.conf.orig
        echo "-h" > ${tmpdir}/img/boot.config
        echo 'console="comconsole"' >> ${tmpdir}/img/boot/loader.conf
fi

if [ ${dopause} -eq 1 ]; then
  echo "Pausing to allow manual review and modification of image file:"
  echo "Image is located in ${tmpdir}/img"
  echo "If you need to fix up ${tmpdir}/img/etc/fstab, now is the time."
  pause
fi


cleanup() {
    umount ${tmpdir}/iso
    mdconfig -d -u ${isodev}
    umount ${tmpdir}/img
    mdconfig -d -u ${imgdev}
    rm -rf ${tmpdir} ${tmpfile}
}

cleanup

ls -lh ${imgoutfile}

echo "To write the image to flash, use dd, for example:"
echo "   dd if=${imgoutfile} of=/dev/da0 bs=4M"
----- cut here ----
-- 
    Clifton Royston  --  cliftonr at iandicomputing.com / cliftonr at lava.net
          I and I Computing * http://www.iandicomputing.com/
 Custom programming, network design, systems and network consulting services