VM deadlock?
John Baldwin
jhb at freebsd.org
Tue Nov 7 18:40:27 UTC 2006
On Sunday 05 November 2006 10:24, Pawel Jakub Dawidek wrote:
> Hi.
>
> We're seeing such a deadlock, seems to be VM related very often. Servers
> are quite loaded, this is FreeBSD 6.2 on SMP Opteron system, but running
> i386 FreeBSD.
>
> Here is some info from one of such machines:
>
> db> ps
> pid ppid pgrp uid state wmesg wchan cmd
> 90346 4316 4316 101 L *Giant 0xc8551ac0 qmail-smtpd
> [...]
> 89997 4316 4316 101 L *vm page 0xcabc9600 qmail-smtpd
> [...]
> 4316 4311 4316 101 LL *vm page 0xcabc9600 qmail-smtpd
> 4315 4311 4315 101 RL drwebd
> [...]
> 34 0 0 0 LL *vm page 0xcabc9600 [pagezero]
> [...]
> 16 0 0 0 LL *vm page 0xcabc9600 [swi1: net]
> [...]
> 14 0 0 0 LL *tcp 0xcabc82c0 [swi4: clock sio]
> [...]
> 3 0 0 0 LL *vm page 0xcabc9600 [g_up]
>
> db> alltrace
> Tracing command drwebd pid 4315 tid 100105 td 0xcac19600
> sched_switch(cac19600,0,2) at sched_switch+0x14b
> mi_switch(2,0) at mi_switch+0x1ba
> critical_exit(0,cac19600,ca0b5688,8246000,0,...) at critical_exit+0x9d
> intr_execute_handlers(c844b0c8,f8211b90,34,f8211be0,c06590b3,...) at
intr_execute_handlers+0x10c
> lapic_handle_intr(31) at lapic_handle_intr+0x2e
> Xapic_isr1() at Xapic_isr1+0x33
> --- interrupt, eip = 0xc06676c4, esp = 0xf8211bd4, ebp = 0xf8211be0 ---
> pmap_pte_quick(ca0b5688,8246000) at pmap_pte_quick+0x90
> pmap_copy(ca0b5688,c88d1ea0,81a1000,a18000,81a1000) at pmap_copy+0x27c
> vm_map_copy_entry(c88d1de0,ca0b55c8,cabc4374,cac161dc) at
vm_map_copy_entry+0x14c
> vmspace_fork(c88d1de0,4,c854b860,c8823d00,f8211cbc,...) at
vmspace_fork+0x22a
> vm_forkproc(cac19600,c854b860,c854c900,14) at vm_forkproc+0xb3
> fork1(cac19600,14,0,f8211cd4) at fork1+0x1169
> fork(cac19600,f8211d04) at fork+0x18
> syscall(3b,3b,bfbf003b,4,81bba00,...) at syscall+0x2bf
> Xint0x80_syscall() at Xint0x80_syscall+0x1f
> --- syscall (2, FreeBSD ELF32, fork), eip = 0x80e493f, esp = 0xbfbfdcac, ebp
= 0xbfbfe148 ---
>
> db> sh allchain
> chain 1:
> thread 100260 (pid 90346, qmail-smtpd) blocked on lock 0xc07177e0 (sleep
mutex) "Giant"
> thread 100133 (pid 89997, qmail-smtpd) blocked on lock 0xc0722700 (sleep
mutex) "vm page queue mutex"
> thread 100105 (pid 4315, drwebd) is on a run queue
> chain 2:
> thread 100098 (pid 4316, qmail-smtpd) blocked on lock 0xc0722700 (sleep
mutex) "vm page queue mutex"
> thread 100105 (pid 4315, drwebd) is on a run queue
> chain 3:
> thread 100031 (pid 34, pagezero) blocked on lock 0xc0722700 (sleep
mutex) "vm page queue mutex"
> thread 100105 (pid 4315, drwebd) is on a run queue
> chain 4:
> thread 100015 (pid 3, g_up) blocked on lock 0xc0722700 (sleep mutex) "vm
page queue mutex"
> thread 100105 (pid 4315, drwebd) is on a run queue
> chain 5:
> thread 100002 (pid 14, swi4: clock sio) blocked on lock 0xc072146c (sleep
mutex) "tcp"
> thread 100000 (pid 16, swi1: net) blocked on lock 0xc0722700 (sleep
mutex) "vm page queue mutex"
> thread 100105 (pid 4315, drwebd) is on a run queue
> db> sh locktree
> db> sh turnstile
>
> Process drwebd (pid 4315) doesn't seem to go any further.
How about 'show allpcpu' to see which threads are running?
--
John Baldwin
More information about the freebsd-current
mailing list