em0: Watchdog timeout -- resetting

[Lev Serebryakov]

Hello, Eugene & Jack.
You wrote 1 февраля 2011 г., 11:23:23:

Eugene wrote:
> You could give a try to netisr parallelism of RELENG_8 instead of POLLING
> (and tune interrupt throttling) if your box does not have lots of dynamic
> interfaces like when using mpd.

Jack wrote:
> I don't test POLLING, sounds like its broken, I don't understand
> why you think you need you need it?  This hardware supports
> MSI why not use it?

  I send one answer to two messages, because data is the same.

  Here it is snapshot of "top -S" with "H" pressed when server sends
1Gbit/s via SMB with polling (Windows'7 client copies 8GiB sparse file to very
fast local disk):


================= POLLING
CPU:  0.5% user,  0.0% nice,  0.6% system,  1.3% interrupt, 98.1% idle
  PID USERNAME     PRI NICE   SIZE    RES STATE   C   TIME   WCPU COMMAND
   11 root         171 ki31     0K    32K CPU1    1  90.1H 100.00% {idle: cpu1}
   11 root         171 ki31     0K    32K RUN     0  82.1H 100.00% {idle: cpu0}
   12 root         -64    -     0K   304K WAIT    0  33:40  0.68% {irq18: uhci2 ehc}
   12 root         -44    -     0K   304K WAIT    1 225:22  0.00% {swi1: netisr 0}
   14 root         -68    -     0K   528K -       1  16:19  0.00% {usbus3}
   12 root         -40    -     0K   304K WAIT    0  14:25  0.00% {swi2: cambio}
   12 root         -64    -     0K   304K WAIT    1  12:50  0.00% {irq22: ahci0}
    4 root          -8    -     0K    16K -       0  12:26  0.00% g_down
================= POLLING
NB: no "smbd" process at all in first 8 positions.
Real speed (accroding to Windows'7 report) ~75MiB/s.


  the same without polling, with net.isr settings:
# sysctl net.isr
net.isr.numthreads: 1
net.isr.maxprot: 16
net.isr.defaultqlimit: 256
net.isr.maxqlimit: 10240
net.isr.bindthreads: 0
net.isr.maxthreads: 1
net.isr.direct: 0
net.isr.direct_force: 0

================= INTR - ISR.DIRECT=0
CPU:  3.8% user,  0.0% nice, 26.5% system,  6.6% interrupt, 63.2% idle
  PID USERNAME     PRI NICE   SIZE    RES STATE   C   TIME   WCPU COMMAND
   11 root         171 ki31     0K    32K RUN     0  82.1H 83.59% {idle: cpu0}
   11 root         171 ki31     0K    32K RUN     1  90.1H 64.06% {idle: cpu1}
33873 root          72    0 28912K  5432K select  0   0:28 34.96% smbd
   12 root         -44    -     0K   304K WAIT    0 225:29  9.18% {swi1: netisr 0}
    0 root         -68    0     0K   128K -       1   0:02  6.30% {em0 taskq}
   12 root         -68    -     0K   304K WAIT    0   0:00  1.56% {irq20: em0 fwohc}
    7 root          44    -     0K    16K psleep  0   3:12  0.39% pagedaemon
   12 root         -64    -     0K   304K WAIT    1  33:41  0.20% {irq18: uhci2 ehc}
   14 root         -68    -     0K   528K -       0  16:19  0.00% {usbus3}
   12 root         -40    -     0K   304K WAIT    0  14:25  0.00% {swi2: cambio}
================= INTR - ISR.DIRECT=0
Real speed (accroding to Windows'7 report) ~85MiB/s.

  the same without polling, with net.isr settings:
# sysctl net.isr
net.isr.numthreads: 1
net.isr.maxprot: 16
net.isr.defaultqlimit: 256
net.isr.maxqlimit: 10240
net.isr.bindthreads: 0
net.isr.maxthreads: 1
net.isr.direct: 1
net.isr.direct_force: 1

================= INTR - ISR.DIRECT=1
CPU:  2.8% user,  0.0% nice, 30.1% system,  1.7% interrupt, 65.4% idle
  PID USERNAME     PRI NICE   SIZE    RES STATE   C   TIME   WCPU COMMAND
   11 root         171 ki31     0K    32K RUN     1  90.2H 89.36% {idle: cpu1}
   11 root         171 ki31     0K    32K RUN     0  82.2H 67.87% {idle: cpu0}
33873 root         103    0 28912K  5424K CPU0    0   0:51 33.98% smbd
    0 root         -68    0     0K   128K -       1   0:06 12.70% {em0 taskq}
   12 root         -68    -     0K   304K WAIT    0   0:01  1.66% {irq20: em0 fwohc}
    7 root          45    -     0K    16K psleep  0   3:12  0.78% pagedaemon
   12 root         -64    -     0K   304K WAIT    0  33:42  0.20% {irq18: uhci2 ehc}
   12 root         -44    -     0K   304K WAIT    1 225:33  0.00% {swi1: netisr 0}
   14 root         -68    -     0K   528K -       1  16:20  0.00% {usbus3}
   12 root         -40    -     0K   304K WAIT    0  14:25  0.00% {swi2: cambio}
================= INTR - ISR.DIRECT=1
Real speed (accroding to Windows'7 report) ~101MiB/s.

  I've re-created file to flush caches on both sides between trys.

-- 
// Black Lion AKA Lev Serebryakov <lev at FreeBSD.org>