Troubles with em on FreeBSD 7

[Oleksandr Samoylyk]

Jack Vogel wrote:
> Oh, I just had a thought, increase the RX processing limit,
> that only allows you to process 100 packets in one pass.
> 
> First change it to 250 and see what it does, you might
> also set it to -1 which will allow you to process til you
> drain the ring, the risk is that you cause other problems
> by doing that, but heck at this point anything is worth
> trying, right?
> 

last pid: 55326;  load averages:  1.40,  1.24,  1.05 
                                            up 0+06:56:52  08:32:41
83 processes:  10 running, 60 sleeping, 13 waiting
CPU states:  0.5% user,  0.0% nice, 10.5% system,  1.1% interrupt, 87.9% 
idle
Mem: 42M Active, 265M Inact, 272M Wired, 244K Cache, 214M Buf, 7338M Free
Swap: 4096M Total, 4096M Free

   PID USERNAME  THR PRI NICE   SIZE    RES STATE  C   TIME   WCPU COMMAND
    12 root        1 171 ki31     0K    16K CPU6   6 413:05 100.00% 
idle: cpu6
    16 root        1 171 ki31     0K    16K RUN    2 401:28 100.00% 
idle: cpu2
    15 root        1 171 ki31     0K    16K CPU3   3 402:45 96.09% idle: 
cpu3
    17 root        1 171 ki31     0K    16K CPU1   1 398:26 92.68% idle: 
cpu1
    14 root        1 171 ki31     0K    16K CPU4   4 401:24 90.28% idle: 
cpu4
    18 root        1 171 ki31     0K    16K CPU0   0 382:00 89.26% idle: 
cpu0
    13 root        1 171 ki31     0K    16K CPU5   5 392:57 81.40% idle: 
cpu5
    11 root        1 171 ki31     0K    16K CPU7   7 378:44 59.47% idle: 
cpu7
    30 root        1 -68    -     0K    16K CPU7   7  37:24 44.78% em1 taskq
    31 root        1 -68    -     0K    16K -      5  23:00 18.26% em2 taskq
    19 root        1 -44    -     0K    16K WAIT   3  35:45 14.60% swi1: net
    32 root        1 -68    -     0K    16K -      0  19:51  7.08% em3 taskq
   722 root        2   5    0 68280K 21652K select 2   0:00  1.46% mpd5
    20 root        1 -32    -     0K    16K WAIT   2   3:56  0.00% swi4: 
clock
    29 root        1 -68    -     0K    16K -      6   2:53  0.00% em0 taskq
    22 root        1  44    -     0K    16K -      1   0:49  0.00% yarrow

I've added more NICs.
The thing is that the most busiest interface is em3 (uplink) and it's 
just 7%.

# netstat -I em0 -w 1 (just for the way back traffic)
             input          (em0)           output
    packets  errs      bytes    packets  errs      bytes colls
         13     0       2344       8408     0    4290811     0
         20     0       2232       8147     0    4179926     0
         12     0       1358       8204     0    3984088     0
         33     0       5660       7982     0    3944704     0
         25     0       3165       8183     0    4226968     0
         10     0       1012       8620     0    4288722     0
         25     0       2822       8283     0    4220019     0
^C

# netstat -I em1 -w 1
             input          (em1)           output
    packets  errs      bytes    packets  errs      bytes colls
       7137     0    2618138          0     0          0     0
       6772     0    2565885          0     0          0     0
       6982     0    2430458          0     0          0     0
       6758     0    2459663          0     0          0     0
       7036     0    2665113          0     0          0     0
^C

# netstat -I em2 -w 1
             input          (em2)           output
    packets  errs      bytes    packets  errs      bytes colls
       1237     0     327276          0     0          0     0
       1158     0     328459          0     0          0     0
       1243     0     346992          0     0          0     0
       1076     0     323918          0     0          0     0
       1242     0     363008          0     0          0     0
^C

# netstat -I em3 -w 1
             input          (em3)           output
    packets  errs      bytes    packets  errs      bytes colls
       5681     0    3822499       6233     0    2443102     0
       5958     0    4233009       6692     0    2354643     0
       6202     0    4296877       6432     0    2455513     0
       6117     0    4146643       6419     0    2327346     0
       6324     0    4128040       6695     0    2468243     0
       5856     0    4052255       6252     0    2445518     0
^C

Why so?

-- 
  Oleksandr Samoylyk
  OVS-RIPE