serious networking (em) performance (ggate and NFS) problem
Daniel Eriksson
daniel_k_eriksson at telia.com
Thu Nov 18 09:49:15 PST 2004
M. Warner Losh wrote:
> Also, make sure that you aren't sharing interrupts between
> GIANT-LOCKED and non-giant-locked cards. This might be exposing bugs
> in the network layer that debug.mpsafenet=0 might correct. Just
> noticed that our setup here has that setup, so I'll be looking into
> that area of things.
I have a Tyan Tiger MPX board (dual AthlonMP) that has two 64bit PCI slots.
I have an Adaptec 29160 and a dual port Intel Pro/1000 MT plugged into those
slots.
As can be seen from the vmstat -i output below, em1 shares ithread with
ahc0. This is with ACPI disabled. With ACPI enabled all devices get their
own ithread (I think, not 100% sure). However, because of some hardware
problem (interrupt routing?), em0 interrupts will somehow leak into
atapci1+, generating a higher interrupt load. I'm not sure how expensive
this is.
Finally, my question. What would you recommend:
1) Run with ACPI disabled and debug.mpsafenet=1 and hope that the mix of
giant-safe and giant-locked (em and ahc) doesn't trigger any bugs. This is
what I currently do.
2) Run with ACPI disabled and debug.mpsafenet=0 and accept lower network
performance (it is a high-traffic server, so I'm not sure this is a valid
option).
3) Run with ACPI enabled and debug.mpsafenet=1 and accept that em0
interrupts "leak" to the atapci1+ ithread. This I have done in the past.
# vmstat -i
interrupt total rate
irq1: atkbd0 2 0
irq4: sio0 710 0
irq6: fdc0 8 0
irq8: rtc 11470937 127
irq13: npx0 1 0
irq14: ata0 1744545 19
irq15: ata1 1749617 19
irq16: em0 atapci1+ 186062858 2074
irq17: em1 ahc0 27028088 301
irq18: atapci3 7393468 82
irq19: atapci4+ 7129446 79
irq0: clk 179054582 1995
Total 421634262 4699
/Daniel Eriksson
More information about the freebsd-current
mailing list