FreeBSD 9: fdisk -It crashes kernel

Thu Apr 25 17:04:29 UTC 2013

On Apr 25, 2013, at 10:58 AM, Jeremy Chadwick <jdc at koitsu.org> wrote:

> On Thu, Apr 25, 2013 at 09:06:49AM -0500, Guy Helmer wrote:
>> Encountered a surprise when my disk resizing rc.d script caused FreeBSD 9.1-STABLE to crash. I used "fdisk -It ada0" to determine what the available size of the disk (which happened to be the root disk), and on FreeBSD 9.1 the kernel comes crashing down:
>> 
>> + fdisk -It ada0
>> + /rescue/sed -En 's,.*start ([0-9]+).*size ([0-9]+).*,\1 + \2,p'
>> vnode_pager_getpages: I/O read error
>> vm_fault: pager read error, pid 65 (fdisk)
>> pid 65 (fdisk), uid 0: exited on signal 11
>> eval: arithmetic expression: expecting primary: ""
>> Entropy harvesting: point_to_pointeval: date: Device not configured
>> eval: df: Device not configured
>> eval: dmesg: Device not configured
>> cat: /bin/ls: Device not configured
>> kickstart.
>> eval: cannot open /etc/fstab: Device not configured
>> eval: cannot open /etc/fstab: Device not configured
>> eval: swapon: Device not configured
>> Warning! No /etc/fstab: skipping disk checks
>> fstab: /etc/fstab:0: Device not configured
>> 
>> Fatal trap 12: page fault while in kernel mode
>> cpuid = 1; apic id = 01
>> fault virtual address   = 0x0
>> fault code                     = supervisor read, page not present
>> instruction pointer      = 0x20:0xc0825fc4
>> stack pointer               = 0x28:0xc5a088c8
>> frame pointer              = 0x28:0xc5a08914
>> code segment            = base 0x0, limit 0xfffff, type 0x1b
>>                                     = DLP 0, pres 1, def32 1, gran 1
>> processor eflags       = interrupt enabled, resume, IOPL = 0
>> current process         = 91 (mount)
>> [ thread pid 91 tid 100056 ]
>> Stopped at  g_access+0x24: mlvl 0(%ebx),%eax
>> db> where
>> Tracing pid 91 tid 100056 td 0xc84c42f0
>> g_access(c8481d34,0,1,1,0,…) at g_access+0x24/frame 0xc5a08914
>> ffs_mount(c8481d34,c0d78380,2,c5a08c00,c829ae6c,…) af ffs_mount+0xf74/frame 0xc5a08a34
>> vfs_donmount(c84c42f0,10000,0,c84cf200,c84cf200,…) at vfs_donmount+0x1423/frame 0xc5a08c24
>> sys_nmount(c84c42f0,c5a08ccc,c5a08cc4,1010006,c5a08d08,…) at sys_nmount+0x7f/frame 0xc5a08c48
>> syscall(c5a08d08) at syscall+0x443/frame 0xc508cfc
>> Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xc5a08cfc
>> --- syscall (378, FreeBSD ELF32, sys_nmount), eip = 0x480d5feb, esp = 0xbfbfce1c, ebp = 0xbfbfd378 ---
>> 
>> I'll fix my script to not do this, but it seems odd that fdisk -It can make the disk "go away".
> 
> Please provide a full, unmodified copy of your script.
> 
> What's confusing to me is that after your sed call (which I don't even
> understand, because it doesn't appear to be operating on anything except
> stdin/stdout, and we don't know what that is -- again, show the script),
> the kernel starts outputting indications that the root disk/filesystem
> or its related metadata disappeared:
> 
>> vnode_pager_getpages: I/O read error
>> vm_fault: pager read error, pid 65 (fdisk)
>> pid 65 (fdisk), uid 0: exited on signal 11
> 
> Except the kernel stack trace indicates something called sys_nmount(),
> which called vfs_donmount(), which called ffs_mount(), which calls
> g_access().  All of those scream to me "someone tried to mount
> something".  fdisk does not do mounting.

Right, which is why I copied the entire screen output -- it appears to me that the rc scripts had stumbled on until the kernel panicked.

> 
> fdisk also shouldn't be writing to LBA 0 (the MBR) if you used -I -t.
> I've been staring at fdisk.c for about 20 minutes now and I can't work
> out a situation where -I -t would cause the MBR to be rewritten
> actively.
> 
> The only GEOM calls I see in fdisk.c that would get called are
> g_device_path(), g_open(), and g_close().  Actual device I/O uses read()
> and write() (only in write_s0() which shouldn't be called).
> 
> Furthermore, GEOM has foot-shooting-prevention mechanisms in place (I'm
> talking about kern.geom.debugflags) to keep LBA 0 from being modified.
> Is your script setting that sysctl to 16/0x10 blindly?  Ahem.

No. The script is intended only to work for drives other than the one containing the boot partition.

> 
> It would also help if you could state exactly what 9.1-STABLE source
> you're using; if using svn provide revision (rXXXXXX), else provide
> uname -a output.

rev 249788

> 
> Finally: I would suggest using gpart(8) instead going forward.  This is
> a separate recommendation though; if somehow I'm overlooking something
> in fdisk.c where writes to LBA 0 really do happen, then that needs to
> get fixed.  But gpart(8) is what you should use in general these days
> anyway.
> 

Seems like gpart was giving me some frustration with earlier versions of FreeBSD (7, I think) so I went with fdisk instead. Might work OK now...

I have included the full script below.

#!/bin/sh
#
# PROVIDE: growfs_vm
# REQUIRE: root
# BEFORE: disks
#
# Check the disks and partitions on them to see whether partitions
# need to be resized.
#
# Copyright 2011 Palisade Systems, Inc.
#
# $Id: growfs_vm.sh 6687 2012-06-13 19:53:29Z ghelmer $

. /etc/rc.subr

prefix="/usr/local"
name="growfs_vm"
rcvar=$(set_rcvar)

load_rc_config "${name}"

growfs_vm_enable=${growfs_vm_enable-"YES"}

start_cmd="growfs_vm_start"

growfs_vm_min_increase=${growfs_vm_min_increase-"16384"}

SED=/rescue/sed

disk_mounted()
{
    local DISK=$1
    local mounts

    mounts=$(mount | $SED -En -e "s,^/dev/(${DISK})s[0-9]+[a-z]+ on .*,\1,p")
    if [ -z "$mounts" ]; then
	return 1
    fi
    return 0
}

grow_disk()
{
    local DISK=$1
    local physsize
    local curroff
    local currsize
    local delta
    local part
    local bsd_part
    local old_label
    local new_label

    # The last partititon offset + size is the disk size.
    physsize=$(($(fdisk -It $DISK | $SED -En 's,.*start ([0-9]+).*size ([0-9]+).*,\1 + \2,p')))
    eval $(fdisk -v $DISK | $SED -En 's,.*start ([0-9]+).*size ([0-9]+).*,curroff=\1 currsize=\2,p')
    delta=$(($physsize - $curroff - $currsize))

    if [ $delta -le $growfs_vm_min_increase ]; then
	echo "No change in size for disk ${DISK}: delta ${delta} <= min ${growfs_vm_min_increase}"
	return 0
    fi
    # Get the index of the last FreeBSD partition on the disk.
    part=$(gpart show $DISK | $SED -En 's,^ *[0-9]+ +[0-9]+ +(4) +freebsd .*,\1,p')
    if [ -z "$part" ]; then
	part=$(gpart show $DISK | $SED -En 's,^ *[0-9]+ +[0-9]+ +(3) +freebsd .*,\1,p')
    fi
    if [ -z "$part" ]; then
	part=$(gpart show $DISK | $SED -En 's,^ *[0-9]+ +[0-9]+ +(2) +freebsd .*,\1,p')
    fi
    if [ -z "$part" ]; then
	part=$(gpart show $DISK | $SED -En 's,^ *[0-9]+ +[0-9]+ +(1) +freebsd .*,\1,p')
    fi
    # Extend the partition to the end of the disk.
    gpart resize -i $part $DISK
    if [ "$?" -ne 0 ]; then
	echo "gpart resize -i $part $DISK failed"
	return 1
    fi

    # Now, find the last BSD partition and extend it.
    old_label=$(bsdlabel "${DISK}s${part}")
    bsd_part=$(echo "$old_label" | $SED -En 's,^ *(g): +[0-9]+ +[0-9]+ +4\.2BSD .*,\1,p')
    if [ -z "$bsd_part" ]; then
	bsd_part=$(echo "$old_label" | $SED -En 's,^ *(f): +[0-9]+ +[0-9]+ +4\.2BSD .*,\1,p')
    fi
    if [ -z "$bsd_part" ]; then
	bsd_part=$(echo "$old_label" | $SED -En 's,^ *(e): +[0-9]+ +[0-9]+ +4\.2BSD .*,\1,p')
    fi
    if [ -z "$bsd_part" ]; then
	bsd_part=$(echo "$old_label" | $SED -En 's,^ *(d): +[0-9]+ +[0-9]+ +4\.2BSD .*,\1,p')
    fi
    # Extend 'c' pseudo-partition and the last BSD partition
    # to the end of the disk.
    new_label=$(echo "$old_label" | $SED -E \
	-e "s,c: ([0-9]+),c: *," \
	-e "s,${bsd_part}: ([0-9]+),${bsd_part}: *,")
    echo "$new_label" | bsdlabel -R "${DISK}s${part}" /dev/stdin
    if [ "$?" -ne 0 ]; then
	echo "bsdlabel -R ${DISK}s${part} failed"
	return 1
    fi

    # Finally, run growfs on the extended partition.
    growfs -y "${DISK}s${part}${bsd_part}"
    if [ "$?" -ne 0 ]; then
	echo "growfs -y ${DISK}s${part}${bsd_part} failed"
	return 1
    fi
    return 0
}

growfs_vm_start() {
    local maker
    maker=`/bin/kenv smbios.system.maker`
    if [ "$maker"  = "VMware, Inc." ]; then
	echo 'growfs_vm starting.'

	disks=$(gpart show | $SED -En -e 's,=> +[0-9]+ +[0-9]+ +([a-z]+[0-9]+) +MBR .*,\1,p')

	for disk in $disks; do
	    if disk_mounted $disk; then
		echo "Disk ${disk} is mounted: skipping"
	    else
		grow_disk $disk
	    fi
	done

	echo 'growfs_vm finished.'

	mounted_root=$(mount | $SED -En -e 's,(^/dev/|^)([a-z]+[0-9a-z]+) on / .*,\2,p')
	if [ "$mounted_root" = 'cd0' -o "$mounted_root" = 'acd0' ]; then
		# We're running off a boot CD.
	    echo "Finished growfs_vm.  Remove virtual CD and reboot."
	    halt
	fi
    fi
}

run_rc_command "$1"