System hang on shutdown when running freebsd-update

[Kevin Oberman]

On Wed, Oct 15, 2014 at 2:40 AM, Colin Perkins <csp at csperkins.org> wrote:

> On 14 Oct 2014, at 18:09, Kevin Oberman <rkoberman at gmail.com> wrote:
> > I thought that this was just a fluke, but it has now happened three
> times,
> > so I guess it's now out of the "fluke" class.
> >
> > I have upgraded several times recently to each 10.1 BETA and RC. After
> the
> > first install pass t install the kernel and modules, the system shutdown
> > freezes at the very end. I see the buffers synced to the disks and get
> the
> > "All buffers synced" message. Then it just hangs. The disks are not
> marked
> > as clean and are fscked after a reset and boot.
> >
> > There is not much between the "All buffers synced" message and the call
> to
> > vfs_unmountall(), so I suspect it is hanging in that call. I admit that I
> > am pretty much lost whenever I look at the VFS code and I have not put a
> > lot of effort going further. Just hoping that someone familiar with it
> > might have an idea.
> >
> > I have tried several reboots and all run normally. The problem only seems
> > to appear when upgrading the OS. It happened repeatedly when I tried to
> > reboot before doing the second "install" pass of freebsd-update, but not
> > after, so the kernel and world are not in sync. I am baffled as to what
> > could be going on, but it means I need to be at the system (a baby
> server)
> > when I upgrade, but not every time I upgrade. I know it happened on the
> > 10.0-RELEAASE to 10.1-BETA1 and 10.1-RC1 to 10.1-RC2 upgrades.
> >
> > Has anyone else seen this?
>
> I’m seeing the same behaviour, most recently when moving to 10.1-RC1
> (haven’t gone to -RC2 yet). The system is:
>
> FreeBSD 10.1-RC1 #0 r272463: Fri Oct  3 01:47:10 UTC 2014
> root at releng1.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64
> FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512
> CPU: AMD Opteron(TM) Processor 6274                  (2200.05-MHz K8-class
> CPU)
> Origin = "AuthenticAMD"  Id = 0x600f12  Family = 0x15  Model = 0x1
> Stepping = 2
>
> Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT>
>
> Features2=0x1e98220b<SSE3,PCLMULQDQ,MON,SSSE3,CX16,SSE4.1,SSE4.2,POPCNT,AESNI,XSAVE,OSXSAVE,AVX>
> AMD Features=0x2e500800<SYSCALL,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM>
> AMD
> Features2=0x1c9bfff<LAHF,CMP,SVM,ExtAPIC,CR8,ABM,SSE4A,MAS,Prefetch,OSVW,IBS,XOP,SKINIT,WDT,LWP,FMA4,NodeId,Topology,PCXC,PNXC>
> TSC: P-state invariant, performance statistics
> real memory  = 549755813888 (524288 MB)
> avail memory = 534559084544 (509795 MB)
> Event timer "LAPIC" quality 400
> ACPI APIC Table: <041112 APIC1739>
> FreeBSD/SMP: Multiprocessor System Detected: 64 CPUs
> FreeBSD/SMP: 4 package(s) x 16 core(s)
> …
>
> I do have IPMI loaded, unlike the other reports.
>
> --
> Colin Perkins
> https://csperkins.org/
>

Paul Koch replied privately with a pointer to a seemingly unrelated message
he sent to stable last month. Take a look at the several paragraphs at the
end starting with "On a side note". I'm suspicious that the generation of
the large upgrade on /var during the "upgrade" pass is causing the delay.
It fits pretty well and, in normal operation, my server would never see
this issue at all.

https://docs.freebsd.org/cgi/getmsg.cgi?fetch=326083+0+/usr/local/www/db/text/2014/freebsd-stable/20140907.freebsd-stable

Aside from fsyncing the files, I suspect just running "upgrade" waiting for
a long time before doing reboot might prevent it from happening. I is
likely relevant that the single partition on hte system is a 500GB SU+J UFS.

I need to research a bit on how freebsd does things as well as possible
interaction with the large SU+J partition. I was already uncomfortable
about the SU+J but went with it due to the time it would otherwise take to
fsck the 500GB disk.
--
R. Kevin Oberman, Network Engineer, Retired
rkoberman at gmail.com