Using a file-backed swap via md0 results in panic on reboot..

[John Baldwin]

So if you attach an md device to a file and then use that md0 device for swap, 
you will get a panic on shutdown if the swap file was in use.  The reason is 
that we remove swap devices _after_ unmounting the filesystems in boot():

		if (nbusy) {
			/*
			 * Failed to sync all blocks. Indicate this and don't
			 * unmount filesystems (thus forcing an fsck on reboot).
			 */
			printf("Giving up on %d buffers\n", nbusy);
			DELAY(5000000);	/* 5 seconds */
		} else {
			if (!first_buf_printf)
				printf("Final sync complete\n");
			/*
			 * Unmount filesystems
			 */
			if (panicstr == 0)
				vfs_unmountall();
		}
		swapoff_all();
		DELAY(100000);		/* wait for console output to finish */

Is there any reason we can't move the swapoff_all()?  Should we perhaps only 
do it in "clean" case?  Alternatively, should the swapoff_all() during 
shutdown have a special flag so that it doesn't actually read in any data 
from disk and just throws away the data instead?  Sample panic:

Waiting (max 60 seconds) for system process `vnlru' to stop...done
Waiting (max 60 seconds) for system process `syncer' to stop...
Syncing disks, vnodes remaining...2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 
0 0 done
Waiting (max 60 seconds) for system process `bufdaemon' to stop...done
All buffers synced.
Swap device ad0s1b removed.
swap_pager: I/O error - pagein failed; blkno 2097177,size 4096, error 5
panic: swap_pager_force_pagein: read from swap failed
cpuid = 0
KDB: stack backtrace:
panic() at panic+0x2c5
swapoff_one() at swapoff_one+0x5bb
swapoff_all() at swapoff_all+0xe4
boot() at boot+0x871
reboot() at reboot+0x45
syscall() at syscall+0x642
Xfast_syscall() at Xfast_syscall+0xa8

-- 
John Baldwin