FreeBSD-10.3-RELEASE-i386-memstick.img installer changes contents of USB flash drive

[Warren Block]

On Sun, 2 Oct 2016, Ian Smith wrote:

> > A secondary copy of the partition table is saved at the end of a
> > GPT-partitioned drive. This is a problem for GPT when a binary image is
> > written to a device that is not exactly the same size as the image. That is
> > pretty much always the case with installer images.
>
> Indeed.  MBR/BSD images don't have this problem because all metadata is
> at the beginning of and within the image in the case of UFS superblocks.
> So an image can be copied to a device like /dev/da0, in the case of the
> hitherto dedicated drive /dev/da0a, or as easily to a slice like da0s1
> where it becomes da0s1a, as long as there's enough room on the memstick.
>
> A GPT image seems more of a whole-disk image rather than (what equally
> can serve as) a single partition image.  It must have a primary GPT (all
> 34 sectors?) but do you know if these images already contain the full 34
> sector secondary GPT?  i.e, I don't see why only the last sector of the
> copied image file was changed, with no update to any of the prior 33?

Most of those 34 blocks (not a fixed size, by the way) contain partition 
information.  But there are global flags and things like the PMBR which 
could change.  Without knowing exactly what that byte does, it's a 
guess.

Note that we don't know when that change is happening.  It could be the 
machine checking the secondary GPT on the memory stick that is helpfully 
and silently "fixing" something when it recognizes the stick, before it 
is read.  It could be malicious or buggy firmware in any of several 
places, including the UEFI or BIOS or the memory stick micontroller 
itself.  The point is that we don't know that it is the FreeBSD 
installer making that change.

> > gpart can repair
> > ("recover") the backup partition table because it is a copy of the primary
> > partition table.  This might be required during an install to make sure the
> > disk works with particularly strict BIOS or UEFI implementations.  Although
> > it seems like a reboot would be required afterwards.
>
> Hmm, well it's already booted from the memstick and installed the OS, so
> we know the installer disk works in that environment.  I can't see any
> need to be modifying the source disk when it's doing / done its job?

Agreed, but see above.

> And then Perry Hutchinson (cc'd) is having the 'opposite' problem of
> wanting to update the 10.3 i386 memstick he'd booted off with gpart
> recover and gpart add for another partition, apparently successfully
> according to gpart show, only to have the updates disappear on reboot?

Pretty sure that's an unrelated problem I've seen myself:
http://www.wonkity.com/~wblock/docs/html/labels.html

If you change partition labels (or something) and then mount them, that 
information is lost.  If you reboot first, something works right.

I remember a few years back that someone helped me understand what was 
going on, but forget the details (and, unfortunately, who it was).

> Personally I'm wanting to get to a memstick with 2 or 3 bootable slices
> for installers (amd64 and i386) - as least the amd64 one from dvd1 with
> packages - plus a mountable r/w data partition.  Using only native tools
> currently implies using boot0, thus MBR/BSD schema.  Converting a new
> GPT-scheme memstick image to ONE slice on my new 16GB memstick is what I
> need to figure out, possibly by means of using a tar pipeline similar to
> dvd1_to_memstick.sh.  I hope to get time to look at 11.0 make release
> memstick scripts, but I'd appreciate any clues / ideas meanwhile ..

I'd think you could just dd(1) the contents of the single installer disk 
partition into a big-enough MBR partition.  Repeat for each image, then 
use boot0cfg.

It really is a shame we still require separate installers for amd64 and 
i386, a single installer can install either.