CPU underload

[Eugene Grosbein]

On 01.11.2015 01:11, Adrian Chadd wrote:
> Hi,
> 
> I don't see any DELAY in my testing, nor MipsUserIntr, so yeah, I must
> be doing something different.

I had compiled the system using CFLAGS+=-Os trying to minimize code size.
It seems, that was not wise. Now I rebuilt it without -Os and got better numbers:
60MByte/s under same inter-vlan briding test.

However, this time "interrupt storm" stroke again but that was not same bug
but just really higher numbers of interrupts on arge0 due to higher speed.
I raised hw.intr_storm_threshold from default 1000 upto 10000 and "storm" is no more.

"Unaligned Load Word" messages in dmesg I've mentioned before are generated by
running pmcstat. If I do not run it, dmesg is not flooded and its last line is:
"Nov  1 01:43:06 tl-wdr3600 login: ROOT LOGIN (root) ON ttyu0". No port flapping.

CPU utilization has raised too, but idle part is still pretty large
and it is very unstable:

# vmstat 1
procs  memory      page                       disks     faults         cpu
r b w  avm   fre   flt  re  pi  po    fr   sr fl0 md0   in    sy    cs us sy id
0 0 0  58M   93M   297   0   3   0   217    2   0   0 1826  8680   184  5 63 32
0 0 0  58M   93M     0   0   0   0     0    2   0   0 3773   105   138  0 78 22
0 0 0  58M   93M     0   0   0   0     0    2   0   0 2316   105   116  0 62 38
0 0 0  58M   93M     0   0   0   0     0    5   0   0 5786   105   167  0 80 20
0 0 0  58M   93M     0   0   0   0     0    2   0   0 4177   105    95  0 79 21
0 0 0  58M   93M     0   0   0   0     0    2   0   0 3032   105    85  0 81 19
0 0 0  58M   93M     0   0   0   0     0    2   0   0 2831   105   152  0 66 34
0 0 0  58M   93M     0   0   0   0     0    1   0   0 4436   105   147  0 75 25
0 0 0  58M   93M     0   0   0   0     0    2   0   0 2943   105    84  0 75 25
0 0 0  58M   93M     0   0   0   0     0    2   0   0 2886   105   141  0 73 27
0 0 0  58M   93M     0   0   0   0     0    3   0   0 5727   105    50  0 95  5
0 0 0  58M   93M     0   0   0   0     0    2   0   0 3021   105   141  0 64 36
0 0 0  58M   93M     0   0   0   0     0    2   0   0 2824   105   127  0 76 24
0 0 0  58M   93M     0   0   0   0     0    2   0   0 2168   105   153  0 54 46
0 0 0  58M   93M     0   0   0   0     0    2   0   0 4879   105   116  0 83 17
0 0 0  58M   93M     0   0   0   0     0    2   0   0 4341   105   210  0 81 19
0 0 0  58M   93M     0   0   0   0     0    2   0   0 2294   105   127  0 54 46
0 0 0  58M   93M     0   0   0   0     0    2   0   0 4178   109   126  1 79 20
0 0 0  58M   93M     0   0   0   0     0    5   0   0 5048   105   106  0 72 28

pmcstat still shows some DELAY samples (but much less now) and
MipsUserIntr is still here:

PMC: [INSTR_EXECUTED] Samples: 14535 (100.0%) , 0 unresolved

%SAMP IMAGE      FUNCTION             CALLERS
  9.1 kernel     MipsUserIntr
  6.1 kernel     if_inc_counter       vlan_transmit:1.1 bridge_forward:1.1 ...
  3.8 kernel     ether_nh_input       netisr_dispatch_src
  3.7 kernel     bridge_input         ether_nh_input
  3.5 kernel     _bus_dmamap_sync     arge_start_locked:1.7 arge_intr:1.0 ...
  3.3 kernel     _bus_dmamap_load_buf _bus_dmamap_load_mbuf_sg
  3.1 kernel     critical_enter
  3.1 kernel     arge_intr            intr_event_execute_handlers
  2.9 kernel     bridge_rtnode_lookup bridge_rtlookup:1.5 ...
  2.9 kernel     memcpy
  2.8 kernel     arge_start_locked    arge_start
  2.6 libc.so.7  bsearch              pmcstat_symbol_search
  2.2 kernel     critical_exit        if_inc_counter
  2.1 kernel     bridge_forward       bridge_input
  1.9 kernel     bridge_rtnode_addr_c
  1.9 kernel     bus_dmamap_sync_buf  _bus_dmamap_sync
  1.9 kernel     DELAY
  1.8 kernel     arge_newbuf          arge_intr
  1.7 kernel     netisr_dispatch_src  netisr_dispatch
  1.6 kernel     mipsNN_pdcache_inv_r
CONVERSION STATISTICS:g_read          pmcstat_process_log
 #exec/elf                                2
 #samples/total                           261959
 #samples/unknown-function                8887