New FreeBSD snapshots available: stable/10 (20150625 r284813)

[Kurt Lidl]

On 7/1/15 7:41 PM, Glen Barber wrote:
> On Wed, Jul 01, 2015 at 05:56:48PM -0400, Kurt Lidl wrote:
>> On 7/1/15 11:52 AM, Chris Ross wrote:
>>>
>>>> On Jul 1, 2015, at 11:34, Kurt Lidl <lidl at pix.net> wrote:
>>>> I discovered that if I comment out the following lines
>>> >from my /etc/rc.conf, the machine boots reliably:
>>>>
>>>> ifconfig_bge0="DHCP"
>>>> ifconfig_bge0_ipv6="inet6 accept_rtadv"
>>>>
>>>> Of course, with no network connection, it's not a very useful machine,
>>>> but this probably is important to know while debugging the cause of
>>>> the problem.
>>>
>>>    I had realized that bringing the interface up was the trigger for the panic.
>>> An interesting thing, given the above, would be to note whether using
>>> a static IPv4 address (i.e., not DHCP or SYNCDHCP which is what I have),
>>> or whether disabling IPv6 or IPv6 accept_rtadv would make any difference.
>>>
>>>    I’m guessing it won’t matter, but I also have a similar config in my rc.conf,
>>> so testing a wider variety of configs might be of value.
>>
>> To answer the question...
>>
>> If you take out the IPv6 configuration it doesn't panic!
>>
>> (I updated the bug with this information too.)
>>
>
> Kurt, can you re-enable the ipv6 line in rc.conf(5), and add '-tso6' to
> your rc.conf(5) lines?
>
>   ifconfig_bge0="DHCP"
>   ifconfig_bge0_ipv6="inet6 accept_rtadv -tso6"
>
> Glen


I tried this, and it panic'd in the same manner.  (Note - I've upgraded
this machine to the second 10.2-PRELEASE build.)

Writing entropy file:.
Setting hostname: spork.pix.net.
spin lock 0xc0cba338 (smp rendezvous) held by 0xfffff80003eb76d0 (tid 
100339) too long
timeout stopping cpus
panic: spin lock held too long
cpuid = 1
KDB: stack backtrace:
#0 0xc05757c0 at panic+0x20
#1 0xc0559250 at _mtx_lock_spin_failed+0x50
#2 0xc0559318 at _mtx_lock_spin_cookie+0xb8
#3 0xc08d801c at tick_get_timecount_mp+0xdc
#4 0xc05840c8 at binuptime+0x48
#5 0xc08a400c at timercb+0x6c
#6 0xc08d8380 at tick_intr+0x220
Uptime: 23s
Dumping 8192 MB (4 chunks)

I've also seen (now that it's been running a bit longer), a couple of
other occurrences of the "spin lock held too long" panic. So while
having the IPv6 configuration in /etc/rc.conf causes this crash to
occur most of the time on boot, the same crash occurs at other times
too, which don't appear to IPv6 related.

1) when making the requested change, I editted my /etc/rc.conf file,
and then issued "reboot".  The machine panic'd during the reboot
processing:

root at spork:~ # reboot
Jul  2 09:48:53 spork reboot: rebooted by root
Jul  2 09:48:53 spork syslogd: exiting on signal 15
Waiting (max 60 seconds) for system process `vnlru' to stop...done
Waiting (max 60 seconds) for system process `bufdaemon' to stop...done
Waiting (max 60 seconds) for system process `syncer' to stop...
Syncing disks, vnodes remaining...0 0 0 0 done
All buffers synced.
Uptime: 14h34m16s
GEOM_MIRROR: Device gswap: provider mirror/gswap destroyed.
GEOM_MIRROR: Device gswap destroyed.
pid 1 (init), uid 0: exited on signal 4
spin lock 0xc0cba338 (smp rendezvous) held by 0xfffff8000bbbe920 (tid 
100367) too long
timeout stopping cpus
panic: spin lock held too long
cpuid = 1
KDB: stack backtrace:
#0 0xc05757c0 at panic+0x20
#1 0xc0559250 at _mtx_lock_spin_failed+0x50
#2 0xc0559318 at _mtx_lock_spin_cookie+0xb8
#3 0xc08d801c at tick_get_timecount_mp+0xdc
#4 0xc05840c8 at binuptime+0x48
#5 0xc08a400c at timercb+0x6c
#6 0xc08d8380 at tick_intr+0x220
Uptime: 14h34m16s
Automatic reboot in 15 seconds - press a key on the console to abort
Rebooting...
timeout stopping cpus
timeout shutting down CPUs.

SC Alert: Host System has Reset

Note: the "SC Alert:" message comes the Sparc's ALOM management system,
so that's from the hardware directly, not from FreeBSD's kernel.

It didn't crashdump, so I don't have any other backtrace other than
what I just copied here.

2) After I rebooted with the "-tso" flag in place and it crashed,
I booted again, single user, so I could edit the /etc/rc.conf again
and manually did a savecore:

Trying to mount root from zfs:sys/ROOT/default []...
Enter full pathname of shell or RETURN for /bin/sh:
# mount -u /
# zfs mount -a
# savecore /var/crash
savecore: reboot after panic: spin lock held too long
savecore: writing core to /var/crash/vmcore.1
# vi /etc/rc.conf

[edit session elided - I just commented out the IPv6 config]

# exit
Setting hostuuid: 912fab2a-2040-11e5-8852-0003bae0ce07.
Setting hostid: 0x69fb7ee2.
Entropy harvesting: interrupts ethernet point_to_point swi.
Fast boot: skipping disk checks.
Mounting local file systems:.
Writing entropy file:.
Setting hostname: spork.pix.net.
spin lock 0xc0cba338 (smp rendezvous) held by 0xfffff8000d54e000 (tid 
100357) too long
timeout stopping cpus
panic: spin lock held too long
cpuid = 1
KDB: stack backtrace:
#0 0xc05757c0 at panic+0x20
#1 0xc0559250 at _mtx_lock_spin_failed+0x50
#2 0xc0559318 at _mtx_lock_spin_cookie+0xb8
#3 0xc08d801c at tick_get_timecount_mp+0xdc
#4 0xc05840c8 at binuptime+0x48
#5 0xc08a400c at timercb+0x6c
#6 0xc08d8380 at tick_intr+0x220
Uptime: 5m57s
Dumping 8192 MB (4 chunks)

[crashdump followed]