svn commit: r265792 - head/sys/kern

[Adrian Chadd]

This to me reads like "we need to fix pmc's locking so it's not so
terrible on multi-multi-core-machines." :)



-a


On 26 August 2014 11:54, Slawa Olhovchenkov <slw at zxy.spb.ru> wrote:
> On Tue, May 20, 2014 at 09:04:25AM -0700, Adrian Chadd wrote:
>
>> On 20 May 2014 08:41, Slawa Olhovchenkov <slw at zxy.spb.ru> wrote:
>>
>> >> (But if you try it on 10.0 and it changes things, by all means let me know.)
>> >
>> > I am try on 10.0, but not sure about significant improvement (may be
>> > 10%).
>> >
>> > For current CPU (E5-2650 v2 @ 2.60GHz) hwpmc don't working (1. after
>> > collect some data `pmcstat -R sample.out -G out.txt` don't decode any;
>> > 2. kldunload hwpmc do kernel crash) and I can't collect detailed
>> > profile information.
>>
>> Yup. I'm starting to get really ticked off at how pmc logging on
>> multi-core devices just "stops" after a while. I'll talk with other
>> pmc people and see if we can figure out what the heck is going on. :(
>
> Now I can test you work on CPU w/ working pmc.
> Current traffic 16.8 Gbit/s.
> CPU: Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz (2000.04-MHz K8-class CPU)
>
> last pid: 22677;  load averages:  9.12,  9.07,  8.90                                                                                   up 0+04:06:06  22:52:57
> 47 processes:  3 running, 44 sleeping
> CPU 0:  11.8% user,  0.0% nice, 43.1% system,  1.6% interrupt, 43.5% idle
> CPU 1:  11.4% user,  0.0% nice, 51.8% system,  0.0% interrupt, 36.9% idle
> CPU 2:  10.6% user,  0.0% nice, 46.3% system,  0.8% interrupt, 42.4% idle
> CPU 3:  11.8% user,  0.0% nice, 45.1% system,  0.4% interrupt, 42.7% idle
> CPU 4:  13.7% user,  0.0% nice, 43.1% system,  0.0% interrupt, 43.1% idle
> CPU 5:  14.5% user,  0.0% nice, 45.9% system,  0.4% interrupt, 39.2% idle
> CPU 6:   0.0% user,  0.0% nice,  5.5% system, 68.6% interrupt, 25.9% idle
> CPU 7:   0.0% user,  0.0% nice,  4.3% system, 70.2% interrupt, 25.5% idle
> CPU 8:   0.0% user,  0.0% nice,  2.7% system, 69.4% interrupt, 27.8% idle
> CPU 9:   0.0% user,  0.0% nice,  4.7% system, 67.1% interrupt, 28.2% idle
> CPU 10:  0.0% user,  0.0% nice,  3.1% system, 76.9% interrupt, 20.0% idle
> CPU 11:  0.0% user,  0.0% nice,  5.1% system, 58.8% interrupt, 36.1% idle
> Mem: 322M Active, 15G Inact, 96G Wired, 956K Cache, 13G Free
> ARC: 90G Total, 84G MFU, 5690M MRU, 45M Anon, 394M Header, 98M Other
> Swap:
>
>
>
> @ CPU_CLK_UNHALTED_CORE [241440 samples]
>
> 10.59%  [25561]    _mtx_lock_spin_cookie @ /boot/kernel/kernel
>  94.25%  [24092]     pmclog_reserve @ /boot/kernel/hwpmc.ko
>   100.0%  [24092]      pmclog_process_callchain
>    100.0%  [24092]       pmc_process_samples
>     100.0%  [24092]        pmc_hook_handler
>      100.0%  [24092]         hardclock_cnt @ /boot/kernel/kernel
>       100.0%  [24092]          handleevents
>        99.62%  [24001]           timercb
>         100.0%  [24001]            lapic_handle_timer
>        00.38%  [91]              cpu_activeclock
>         100.0%  [91]               cpu_idle
>          100.0%  [91]                sched_idletd
>           100.0%  [91]                 fork_exit
>  03.04%  [777]       callout_lock
>   91.63%  [712]        callout_reset_sbt_on
>    98.60%  [702]         tcp_timer_activate
>     94.87%  [666]          tcp_do_segment
>      100.0%  [666]           tcp_input
>       100.0%  [666]            ip_input
>        100.0%  [666]             netisr_dispatch_src
>         100.0%  [666]              ether_demux
>          100.0%  [666]               ether_nh_input
>           100.0%  [666]                netisr_dispatch_src
>            98.05%  [653]                 ixgbe_rxeof @ /boot/kernel/if_ixgbe.ko
>             87.44%  [571]                  ixgbe_msix_que
>              100.0%  [571]                   intr_event_execute_handlers @ /boot/kernel/kernel
>               100.0%  [571]                    ithread_loop
>                100.0%  [571]                     fork_exit
>             12.56%  [82]                   ixgbe_handle_que @ /boot/kernel/if_ixgbe.ko
>              100.0%  [82]                    taskqueue_run_locked @ /boot/kernel/kernel
>               100.0%  [82]                     taskqueue_thread_loop
>                100.0%  [82]                      fork_exit
>            01.95%  [13]                  tcp_lro_flush
>             92.31%  [12]                   ixgbe_rxeof @ /boot/kernel/if_ixgbe.ko
>              100.0%  [12]                    ixgbe_msix_que
>               100.0%  [12]                     intr_event_execute_handlers @ /boot/kernel/kernel
>                100.0%  [12]                      ithread_loop
>             07.69%  [1]                    tcp_lro_rx
>              100.0%  [1]                     ixgbe_rxeof @ /boot/kernel/if_ixgbe.ko
>               100.0%  [1]                      ixgbe_msix_que
>                100.0%  [1]                       intr_event_execute_handlers @ /boot/kernel/kernel
>     05.13%  [36]           tcp_output
>      100.0%  [36]            tcp_do_segment
>       100.0%  [36]             tcp_input
>        100.0%  [36]              ip_input
>         100.0%  [36]               netisr_dispatch_src
>          100.0%  [36]                ether_demux
>           100.0%  [36]                 ether_nh_input
>            100.0%  [36]                  netisr_dispatch_src
>             100.0%  [36]                   ixgbe_rxeof @ /boot/kernel/if_ixgbe.ko
>              91.67%  [33]                    ixgbe_msix_que
>               100.0%  [33]                     intr_event_execute_handlers @ /boot/kernel/kernel
>                100.0%  [33]                      ithread_loop
>              08.33%  [3]                     ixgbe_handle_que @ /boot/kernel/if_ixgbe.ko
>               100.0%  [3]                      taskqueue_run_locked @ /boot/kernel/kernel
>                100.0%  [3]                       taskqueue_thread_loop