newfs_msdos behaviour change between 4.x and 5?

[Bruce Evans]

On Mon, 19 Apr 2004, Geoff Buckingham wrote:

> left out:
>
> fdisk -B -b /boot/cpq-mbr
>
> which uses an  MBR captured from a Compaq server  years ago.
>
> On Mon, Apr 19, 2004 at 02:22:42PM +0000, Geoff Buckingham wrote:
> >
> > Under 4.x the following produces a bootable dos fat partition:
> >
> > newfs_msdos -B /boot/win28.b -S 512 -r 32 -i l -c 32 -n 2 -k 6 -m 0xf8 /dev/da0s1
> > mount_msdos -l /dev/da0s1 /dest
> > cp IO.SY /dest/
> > cp MSDOS.SYS /dest/
> > cp COMMAND.COM /dest/
> >
> > Where /boot/win28.b is a dd'd copy of the first 28 512 byte
> > sectors of a bootable win95 sr2 partition and IO.SYS,
> > MSDOS.SYS and COMMAND.COM come from the same FS.
> >
> > (Obviosley the da0s1 need to be created matching the BIOS's
> > idea of the GEOMTREY)
> >
> > However under 5.2.1 the above produces a non-bootable fs
> >
> > Any ideas where I should be looking for the cause of this?

Look no further than GEOM.  GEOM has new ioctls which are supposed to
make things easier for applications like newfs_msdos, but they mostly
do the wrong things so they actually make things harder.  The output
of certain sysctls must now be parsed, but this would be harder than
what newfs_msdos used to do and is not done; instead, newfs_msdos just
uses the wrong ioctls, except for floppies (see below).

Some details:

The following ioctls do the wrong things:
DIOCGWHEADS, DIOCFWSECTORS:
Like their name suggests, these ioctls return the firmware's idea of
the number of heads and sectors.  But newfs_msdos and most other
applications that need to know the disk geometry don't care about the
firmware's idea.  They need to know the BIOS's idea.  This is currently
unavailable except by asking the user to type it in.  Typing it in for
newfs_msdos is done by specifying the "-h heads" and "-u sectors/track"
parameters on the command line.  You already specify lots of parameters,
so you can probably specify these too without much pain.  Use
"newfs -N ..." under RELENG_4 to determine the working parameters and
the same under -current to check them.

The following ioctls are sometimes not available:
DIOCGWHEADS, DIOCFWSECTORS:
These are just missing for the floppy driver, although disk geometry is
actually still relevant for floppies and these ioctls are easier to
support in the floppy driver than in most drivers.  newfs_msdos has a
workaround; it uses the floppy-specific FD_GTYPE ioctl if it works.

The following ioctls work correctly:
DIOCGSECTORSIZE, DIOCGMEDIASIZE:
I've only missed these using current binaries with non-current kernels.

The follow ioctls are never available:
DIOCGHEADS, DIOCSECPERTRACK, DIOCGMEDIAOFFSET:
The first 2 of these should return the BIOS number of heads and
sectors/ track.  newfs_msdos should use these instead of DIOCGFWHEADS,
DIOCGFWSECTORS.  DIOCGMEDIAOFFSET should return the offset in sectors
of the (sub)device from the beginning of the whole disk device.
newfs_msdos should use this to determine the default for its "-o hidden"
parameter.  In RELENG_4, newfs_msdos has large code to determine this
value correctly.  In -current, it just uses 0.  Unlike the first 2,
this ioctl is not generally useful.  This bug can be worked around by
typing in the value, but the value is not so easy to determine as the
geometry values and doesn't seem to be so critical (but I fear that a
wrong offset could cause wrong parts of the disk to be written).

Some consequences of getting the geometry and media offset wrong:
- nonbootable partitions under old versions of DOS/Windows.  I tried
  versions 4.01, 6.22 and W95.  All had no problems accessing the
  file system (newly created with defaults with a size of 133MB) on
  or running chkdsk.  6.22 and W95's scandisk reported an unrelated
  problem.  Running "sys" to create a bootable partition gave an
  unbootable partition when done under 4.01 and 6.22, but worked
  under W95.  So I think there is only a problem in the bootstrap,
  and then only for old bootstraps including the ones in 4.01, 6.22
  and your old Compaq one.  I haven't tested all the combinations
  with file systems created with correct geometry and forget if
  one worked.

- wrong geometry in fdisk(8).  The most common geometry these days are
  probably H=255/S=63 for the active (BIOS) geoemtry and H=16/S=63 for
  the firmware geometry.  I always use the former, but the latter is
  the firmware geometry on all ATA disks new enough to be larger than
  about 8GB.  fdisk under my de-GEOMed version of -current shows the
  same output as in RELENG_4 (because DIOCFWHEADS is actually DIOCHEADS,
  etc.):

% ******* Working on device /dev/ad0 *******
% parameters extracted from in-core disklabel are:
% cylinders=784 heads=255 sectors/track=63 (16065 blks/cyl)

  Here the parameters were actually extracted from the in-core disklabel,
  although fdisk is not entitled to claim this since it got them via
  DIOCFW*.

  The same fdisk binary under -current gives a different geometry that
  is not suitable for use in fdisk:

% ******* Working on device /dev/ad0 *******
% parameters extracted from in-core disklabel are:
% cylinders=13328 heads=15 sectors/track=63 (945 blks/cyl)

  Here there is no in-core disklabel and fdisk's claim about where it got
  the parameters is just wrong.

- wrong geometry in sysinstall(8).  The normal geometry of H=16/S=63 for
  all new ATA disks is incapable of matching sysinstall's anachronistic
  limit on the number of cylinders (C < 65536) on much the same disks
  that have this geometry:
    - CHS = 16384/16/63 for new ATA drives but only works up to about 8GB
    - sysinstall is happy to expand C up to 65536, but CHS = 65536/16/63
      only works up to about 32GB
    - sysinstall is unhappy to expand H up to 256, but CHS = 65536/256/63
      only works up to about 128GB.  The default of H=16 always needs
      expansion, and sysinstall expresses its unhappiness in a long
      false-alarming message; then it calls Sanitize_BioS_Geom() to fix
      up the geometry.  The problem caused by using the firware geometry
      is that this amost always happens.
    - sysinstall is very unhappy to expand C beyound 65536, so on disks
      larger than 128GB it is impossible to avoid the false alarm even
      by manually specifying a correct geometry.  If you start with
      the default you get the false alarm.  Then if you don't accept
      the purported incorrect geometry and enter any other geometry
      manually, the new one is certain to be considered incorrect too
      and you get the false alarm.  Then you eventually give aup an
      accept a purportedly incorrect geoemtry.  Then Sanitize_BioS_Geom()
      also considers the geometry to be insane, but after blundering
      about a bit, it gives up and (at least on i386's) settles on the
      geometry of C=whatever/H=255/S=63.  Here the value for C is just
      the number of sectors divided by (H*S).  This is certain to be
      larger than 65536 for drivers larger than about 128GB, so
      sysinstall considers it to be insane if it checks it again.
      Fortunately, sysinstall doesn't check again unless you back out,
      and the value of C is even less important than the values of H
      and S (it may affect booting ...), and anyway values > 65536
      don't cause any new problems; values > 1024 are already
      unrepresentable in the partition table, and the same kludges for
      not using unrepresentable values work.
  Disclaimer: the above analysis is from reading the code and problem
  reports.  I only use sysinstall for testing, and haven't used it for
  about 5 years.

Summary: to create bootable and/or trustworthy file systems using
newfs_msdos under -current, you must now specify the correct -h, -u
and -o parameters on the command line.

Bruce