Hard lockup on 5.3-STABLE
James Welsh
jwelsh at digitalspy.co.uk
Tue Feb 8 19:56:30 PST 2005
Hi,
Our AMD Opteron server is locking up every few hours - no core dumps being
made, and by the look of the output in /var/log/messages, whatever is
occurring is fairly severe:
Feb 9 03:29:26 pollux kernel:
Feb 9 03:29:26 pollux kernel:
Feb 9 03:29:26 pollux kernel: Fatal trap 12: page fault while in kernel
mode
Feb 9 03:29:26 pollux kernel: cpuid = 1; apic id = 01
Feb 9 03:29:26 pollux kernel: fault virtual address = 0x28
Feb 9 03:29:26 pollux kernel: fault code = supervis
Feb 9 03:29:26 pollux kernel: or
Feb 9 03:29:26 pollux kernel: wr
Feb 9 03:29:26 pollux kernel: it
Feb 9 03:29:26 pollux kernel: e
Feb 9 03:29:26 pollux kernel: , page
Feb 9 03:29:26 pollux kernel: n
Feb 9 03:29:26 pollux kernel: ot pr
Feb 9 03:29:26 pollux kernel: esent
Feb 9 03:29:26 pollux kernel:
Feb 9 03:29:26 pollux kernel: in
Feb 9 03:29:26 pollux kernel: st
The machine then reboots.
In case it helps narrow the problem down, lots of the following messages are
also being logged to /var/log/messages:
Feb 8 16:47:27 pollux kernel: Limiting closed port RST response from 313 to
200 packets/sec
Feb 8 16:47:28 pollux kernel: Limiting closed port RST response from 281 to
200 packets/sec
Feb 8 16:47:30 pollux kernel: Limiting closed port RST response from 279 to
200 packets/sec
Feb 8 16:47:31 pollux kernel: Limiting closed port RST response from 265 to
200 packets/sec
The server is running MySQL Server 4.1.9 compiled from ports - this happens
regardless of whether it's compiled with process scope threads or not. It
also doesn't seem to be related to load - I was monitoring things in top at
the time of the most recent crash - the output looked like this:
last pid: 3980; load averages: 0.00, 0.00, 0.00
up 0+10:42:49 03:29:25
97 processes: 1 running, 96 sleeping
CPU states: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100%
idle
Mem: 88M Active, 147M Inact, 150M Wired, 1272K Cache, 214M Buf, 7438M Free
Swap: 4096M Total, 4096M Free
PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU CPU COMMAND
520 mysql 96 0 138M 83540K select 1 20:10 0.00% 0.00% mysqld
520 mysql 20 0 138M 83540K kserel 0 20:10 0.00% 0.00% mysqld
520 mysql 20 0 138M 83540K kserel 0 20:10 0.00% 0.00% mysqld
520 mysql 20 0 138M 83540K kserel 0 20:10 0.00% 0.00% mysqld
520 mysql 20 0 138M 83540K kserel 0 20:10 0.00% 0.00% mysqld
520 mysql 20 0 138M 83540K kserel 0 20:10 0.00% 0.00% mysqld
520 mysql 20 0 138M 83540K kserel 0 20:10 0.00% 0.00% mysqld
520 mysql 20 0 138M 83540K kserel 0 20:10 0.00% 0.00% mysqld
520 mysql 20 0 138M 83540K kserel 0 20:10 0.00% 0.00% mysqld
520 mysql 20 0 138M 83540K kserel 0 20:10 0.00% 0.00% mysqld
520 mysql 20 0 138M 83540K kserel 0 20:10 0.00% 0.00% mysqld
520 mysql 20 0 138M 83540K kserel 0 20:10 0.00% 0.00% mysqld
520 mysql 20 0 138M 83540K kserel 0 20:10 0.00% 0.00% mysqld
520 mysql 20 0 138M 83540K kserel 0 20:10 0.00% 0.00% mysqld
520 mysql 20 0 138M 83540K kserel 0 20:10 0.00% 0.00% mysqld
520 mysql 20 0 138M 83540K kserel 0 20:10 0.00% 0.00% mysqld
520 mysql 20 0 138M 83540K kserel 0 20:10 0.00% 0.00% mysqld
520 mysql 20 0 138M 83540K kserel 0 20:10 0.00% 0.00% mysqld
520 mysql 20 0 138M 83540K kserel 0 20:10 0.00% 0.00% mysqld
520 mysql 20 0 138M 83540K kserel 0 20:10 0.00% 0.00% mysqld
520 mysql 20 0 138M 83540K kserel 0 20:10 0.00% 0.00% mysqld
520 mysql 20 0 138M 83540K kserel 0 20:10 0.00% 0.00% mysqld
520 mysql 20 0 138M 83540K kserel 0 20:10 0.00% 0.00% mysqld
520 mysql 20 0 138M 83540K kserel 0 20:10 0.00% 0.00% mysqld
520 mysql 20 0 138M 83540K kserel 0 20:10 0.00% 0.00% mysqld
520 mysql 20 0 138M 83540K kserel 0 20:10 0.00% 0.00% mysqld
520 mysql 20 0 138M 83540K kserel 0 20:10 0.00% 0.00% mysqld
520 mysql 20 0 138M 83540K kserel 0 20:10 0.00% 0.00% mysqld
520 mysql 20 0 138M 83540K sigwai 1 20:10 0.00% 0.00% mysqld
520 mysql 20 0 138M 83540K ksesig 1 20:10 0.00% 0.00% mysqld
520 mysql 20 0 138M 83540K kserel 1 20:10 0.00% 0.00% mysqld
520 mysql 20 0 138M 83540K kserel 0 20:10 0.00% 0.00% mysqld
520 mysql 20 0 138M 83540K kserel 0 20:10 0.00% 0.00% mysqld
520 mysql 20 0 138M 83540K kserel 0 20:10 0.00% 0.00% mysqld
520 mysql 20 0 138M 83540K kserel 0 20:10 0.00% 0.00% mysqld
520 mysql 20 0 138M 83540K kserel 0 20:10 0.00% 0.00% mysqld
520 mysql 20 0 138M 83540K kserel 0 20:10 0.00% 0.00% mysqld
520 mysql 20 0 138M 83540K kserel 0 20:10 0.00% 0.00% mysqld
520 mysql 20 0 138M 83540K kserel 0 20:10 0.00% 0.00% mysqld
520 mysql 20 0 138M 83540K kserel 0 20:10 0.00% 0.00% mysqld
520 mysql 20 0 138M 83540K kserel 0 20:10 0.00% 0.00% mysqld
520 mysql 20 0 138M 83540K kserel 0 20:10 0.00% 0.00% mysqld
520 mysql 20 0 138M 83540K kserel 0 20:10 0.00% 0.00% mysqld
520 mysql 20 0 138M 83540K kserel 0 20:10 0.00% 0.00% mysqld
520 mysql 20 0 138M 83540K kserel 0 20:10 0.00% 0.00% mysqld
520 mysql 20 0 138M 83540K kserel 0 20:10 0.00% 0.00% mysqld
520 mysql 20 0 138M 83540K kserel 0 20:10 0.00% 0.00% mysqld
520 mysql 20 0 138M 83540K kserel 0 20:10 0.00% 0.00% mysqld
520 mysql 20 0 138M 83540K kserel 0 20:10 0.00% 0.00% mysqld
520 mysql 20 0 138M 83540K kserel 0 20:10 0.00% 0.00% mysqld
520 mysql 20 0 138M 83540K kserel 0 20:10 0.00% 0.00% mysqld
520 mysql 20 0 138M 83540K kserel 0 20:10 0.00% 0.00% mysqld
520 mysql 20 0 138M 83540K kserel 0 20:10 0.00% 0.00% mysqld
520 mysql 20 0 138M 83540K kserel 0 20:10 0.00% 0.00% mysqld
Server:
Dual AMD Opteron 248 2.2GHz
8GB RAM
Tyan S2882 Thunder K86 motherboard
Running FreeBSD 5.3-STABLE
Any help very gratefully received.
Best regards,
James Welsh
More information about the freebsd-amd64
mailing list