freebsd-update and hang during reboot

[Kevin Oberman]

On Mon, Feb 9, 2015 at 9:08 AM, Ian Lepore <ian at freebsd.org> wrote:

> On Mon, 2015-02-09 at 11:41 -0500, Kurt Lidl wrote:
> > Joel wrote:
> > > Hi,
> > >
> > > Just about every machine I have seems to hang after running
> freebsd-update and doing a reboot. The last message on the screen is "All
> buffers synced” and it just freezes.
> > >
> > > This happens when doing a freebsd-update and going from 10.0 to 10.1,
> but also when doing a fresh 10.1 install and using freebsd-update to get
> the latest -pX security patches. As soon as I reboot the machine, it hangs.
> > >
> > > I’ve tried it on several different HP ProLiant models, on Intel NUCs
> and on VMware virtual machines. Same phenomenon everywhere. It’s really
> easy to trigger: just install 10.1, use default settings everywhere,
> freebsd-update fetch/install, shutdown -r now and BOOM. It hangs. I think
> I’ve seen it on
> > >
> > >
> > >
> > >
> > > 30 servers or so now.
> > >
> > > Everything works like it should after the initial hang tough - no
> matter how many times I reboot it completes the reboot cycle just fine.
> > >
> > > I’ve seen several people (mostly on IRC) mention this problem, but no
> solution.
> > >
> > > Is anyone working on fixing this?
> >
> > I ran into this problem in spades when upgrading a set of servers from
> > FreeBSD 9.0 to 9.1.  I happened consistently.  Normal reboots worked,
> > but when going from 9.0 to 9.1, it *ALWAYS* hung, and it always hung
> > at the same place, after printing the "All buffers synced" message.
> >
> > I ultimately determined that if I did the following, rather than
> > just a "reboot" or "shutdown -r now 'FreeBSD 9.1-RELEASE upgrade'",
> > it would consistently AVOID the hang:
> >
> > sync ; sync ; sync ; shutdown -o -n -r now "FreeBSD 9.1 install"
> >
> > Your mileage may vary, but you don't have a lot to lose by trying it.
> >
> > -Kurt
> >
>
> That is just bad advice.  sync(1) does not g'tee that all data has been
> written, no matter how many times you type it.  shutdown -n tells the
> system to abandon unwritten data.  All in all, this is a recipe for
> silent filesystem corruption.  Using it after an update is just asking
> to have a mix of old and new files on the system after the reboot.
>
> A more robust workaround would be to "mount -r" on all filesystems
> before invoking the shutdown (even a shutdown -n should be safe after
> everything has been remounted readonly).  If the mount -r hangs on one
> of the filesystems, then you've probably got a clue as to where a normal
> shutdown is hanging.
>
> -- Ian


I agree, Ian. This is especially true for any system running an active
database.

By the way, the purpose of typing "sync" three times was to spend the time
typing it. Putting it on a single line completely eliminates the advantage
of three "sync"s.

I have had success with a different method:
# shutdown now
Wait 30 seconds
# reboot

This has always worked for my to this point, but I can't promise anything.
I have only one system running a RELEASE these days. I'd be far more
certain if I still had the 20-30 systems I had when I retired.

--
Kevin Oberman, Network Engineer, Retired
E-mail: rkoberman at gmail.com