freebsd-update and hang during reboot

[Ian Lepore]

On Mon, 2015-02-09 at 11:41 -0500, Kurt Lidl wrote:
> Joel wrote:
> > Hi,
> >
> > Just about every machine I have seems to hang after running freebsd-update and doing a reboot. The last message on the screen is "All buffers synced” and it just freezes.
> >
> > This happens when doing a freebsd-update and going from 10.0 to 10.1, but also when doing a fresh 10.1 install and using freebsd-update to get the latest -pX security patches. As soon as I reboot the machine, it hangs.
> >
> > I’ve tried it on several different HP ProLiant models, on Intel NUCs and on VMware virtual machines. Same phenomenon everywhere. It’s really easy to trigger: just install 10.1, use default settings everywhere, freebsd-update fetch/install, shutdown -r now and BOOM. It hangs. I think I’ve seen it on
> >
> >
> >
> >
> > 30 servers or so now.
> >
> > Everything works like it should after the initial hang tough - no matter how many times I reboot it completes the reboot cycle just fine.
> >
> > I’ve seen several people (mostly on IRC) mention this problem, but no solution.
> >
> > Is anyone working on fixing this?
> 
> I ran into this problem in spades when upgrading a set of servers from
> FreeBSD 9.0 to 9.1.  I happened consistently.  Normal reboots worked,
> but when going from 9.0 to 9.1, it *ALWAYS* hung, and it always hung
> at the same place, after printing the "All buffers synced" message.
> 
> I ultimately determined that if I did the following, rather than
> just a "reboot" or "shutdown -r now 'FreeBSD 9.1-RELEASE upgrade'",
> it would consistently AVOID the hang:
> 
> sync ; sync ; sync ; shutdown -o -n -r now "FreeBSD 9.1 install"
> 
> Your mileage may vary, but you don't have a lot to lose by trying it.
> 
> -Kurt
> 

That is just bad advice.  sync(1) does not g'tee that all data has been
written, no matter how many times you type it.  shutdown -n tells the
system to abandon unwritten data.  All in all, this is a recipe for
silent filesystem corruption.  Using it after an update is just asking
to have a mix of old and new files on the system after the reboot.

A more robust workaround would be to "mount -r" on all filesystems
before invoking the shutdown (even a shutdown -n should be safe after
everything has been remounted readonly).  If the mount -r hangs on one
of the filesystems, then you've probably got a clue as to where a normal
shutdown is hanging.

-- Ian