svn commit: r285051 - head/sys/netinet
Ermal Luçi
eri at freebsd.org
Mon Aug 3 15:05:43 UTC 2015
Hello Olivier,
its strange seeing so much contention on the arp tables on your PMC stats.
Do you run ping(to prepopulate arp) or static arp to remove the noise from
that interaction?
Also do you run with flowtable active?
On Mon, Aug 3, 2015 at 3:06 PM, Olivier Cochard-Labbé <olivier at cochard.me>
wrote:
> On Tue, Jul 28, 2015 at 2:42 PM, Gleb Smirnoff <glebius at freebsd.org>
> wrote:
>
>>
>> Here is suggested patch. Ermal and Oliver, can you please test/benchmark
>> it?
>>
>
> Hi,
>
> this patch reduce performanece :-(
>
> Here are the results regarding forwarding:
>
> x r285046.pps.forwarding (IPSEC compiled but not used)
> + r285051.pps.forwarding (IPSEC compiled but not used)
> * r285051-glebius-patched.pps.forwarding (IPSEC compiled but not used)
>
> +--------------------------------------------------------------------------------+
> |xxx xx ****
> * + +++|
> ||_M_A___|
> |_A_| |AM||
>
> +--------------------------------------------------------------------------------+
> N Min Max Median Avg Stddev
> x 5 397733 406951 399300 401613.8 4324.9755
> + 5 478095 482079 480869 480543.6 1666.0282
> Difference at 95.0% confidence
> 78929.8 +/- 4779.72
> 19.6532% +/- 1.19013%
> (Student's t, pooled s = 3277.27)
> * 5 424720 430745 427014 427378.4 2351.7439
> Difference at 95.0% confidence
> 25764.6 +/- 5076.98
> 6.41527% +/- 1.26415%
> (Student's t, pooled s = 3481.1)
>
>
>
> PMC stats during forwarding bench:
> [root at netgate]/data# pmcannotate pmc.forwarding.out
> /data/debug/boot/kernel/kernel.symbols
> CONVERSION STATISTICS:
> #samples/total 33880
> Profile trace for function: __rw_rlock() [6.29%]
> Profile trace for function: ip_forward() [4.68%]
> Profile trace for function: ip_output() [4.64%]
> Profile trace for function: binuptime() [4.05%]
> Profile trace for function: igb_mq_start_locked() [3.79%]
> Profile trace for function: igb_rxeof() [3.46%]
> Profile trace for function: tsc_get_timecount_low_lfence() [3.25%]
> Profile trace for function: ether_output() [3.03%]
> Profile trace for function: rtalloc1_fib() [2.77%]
> Profile trace for function: random_ivy_read() [2.64%]
> Profile trace for function: _rw_runlock_cookie() [2.64%]
> Profile trace for function: ether_nh_input() [2.63%]
> Profile trace for function: ip_input() [2.55%]
> Profile trace for function: key_allocsp_default() [2.39%]
> Profile trace for function: igb_mq_start() [2.39%]
> Profile trace for function: bzero() [2.08%]
> Profile trace for function: uma_zalloc_arg() [1.95%]
> Profile trace for function: memcpy() [1.84%]
> Profile trace for function: _mtx_lock_spin_cookie() [1.83%]
> Profile trace for function: bcopy() [1.76%]
> Profile trace for function: random_harvest_queue() [1.63%]
> Profile trace for function: __mtx_lock_sleep() [1.56%]
> Profile trace for function: uma_zfree_arg() [1.47%]
> Profile trace for function: arpresolve() [1.39%]
> Profile trace for function: in_cksumdata() [1.25%]
> Profile trace for function: bounce_bus_dmamap_load_buffer() [1.22%]
> Profile trace for function: bcmp() [1.13%]
> Profile trace for function: rtalloc_ign_fib() [1.11%]
> Profile trace for function: rn_match() [1.03%]
> Profile trace for function: netisr_dispatch_src() [1.03%]
> Profile trace for function: critical_exit() [1.02%]
> Profile trace for function: bus_dmamap_load_mbuf_sg() [0.87%]
> Profile trace for function: spinlock_exit() [0.79%]
> Profile trace for function: in_cksum_skip() [0.75%]
> Profile trace for function: ip_ipsec_output() [0.75%]
> Profile trace for function: acpi_cpu_c1() [0.74%]
> Profile trace for function: in_broadcast() [0.74%]
> Profile trace for function: spinlock_enter() [0.74%]
> Profile trace for function: igb_refresh_mbufs() [0.71%]
> Profile trace for function: in_lltable_lookup() [0.71%]
> Profile trace for function: ip_fastforward() [0.68%]
> Profile trace for function: m_adj() [0.65%]
> Profile trace for function: ether_demux() [0.65%]
> Profile trace for function: _key_freesp() [0.61%]
> Profile trace for function: lockstat_nsecs() [0.60%]
> Profile trace for function: m_freem() [0.58%]
> Profile trace for function: critical_enter() [0.56%]
> Profile trace for function: m_copydata() [0.55%]
> Profile trace for function: mb_free_ext() [0.54%]
> Profile trace for function: pmap_kextract() [0.50%]
>
>
>
> And about fastforwarding:
>
> x 285046.pps.fastforwarding (IPSEC compiled but not used)
> + 285051.pps.fastforwarding (IPSEC compiled but not used)
> * r285051-glebius-patched.pps.fastforwarding (IPSEC compiled but not used)
>
> +--------------------------------------------------------------------------------+
> |*
> + |
> |* * * * ++ +x
> x+ xx x|
> ||____AM__|
> |_MA__|__AM_| |
>
> +--------------------------------------------------------------------------------+
> N Min Max Median Avg Stddev
> x 5 742683 754709 750940 749015.6 4944.9059
> + 5 736459 747593 738197 740112.4 4511.1611
> Difference at 95.0% confidence
> -8903.2 +/- 6902.82
> -1.18865% +/- 0.921585%
> (Student's t, pooled s = 4733)
> * 5 638356 652855 646668 645263 6599.8604
> Difference at 95.0% confidence
> -103753 +/- 8504.75
> -13.8519% +/- 1.13546%
> (Student's t, pooled s = 5831.39)
>
> And pmcstat during fastforwarding:
>
> [root at netgate]/data# pmcannotate pmc.fastforwading.out
> /data/debug/boot/kernel/kernel.symbols
> CONVERSION STATISTICS:
> #samples/total 37656
> Profile trace for function: __rw_rlock() [9.12%]
> Profile trace for function: ip_fastforward() [5.80%]
> Profile trace for function: binuptime() [5.73%]
> Profile trace for function: rtalloc1_fib() [4.92%]
> Profile trace for function: tsc_get_timecount_low_lfence() [4.71%]
> Profile trace for function: igb_rxeof() [4.51%]
> Profile trace for function: _rw_runlock_cookie() [4.14%]
> Profile trace for function: igb_mq_start_locked() [4.12%]
> Profile trace for function: ether_output() [3.36%]
> Profile trace for function: ether_nh_input() [3.11%]
> Profile trace for function: igb_mq_start() [3.02%]
> Profile trace for function: random_ivy_read() [2.88%]
> Profile trace for function: bzero() [2.70%]
> Profile trace for function: memcpy() [2.54%]
> Profile trace for function: random_harvest_queue() [2.32%]
> Profile trace for function: __mtx_lock_sleep() [1.91%]
> Profile trace for function: bcmp() [1.78%]
> Profile trace for function: _mtx_lock_spin_cookie() [1.73%]
> Profile trace for function: igb_txeof() [1.70%]
> Profile trace for function: rn_match() [1.69%]
> Profile trace for function: bounce_bus_dmamap_load_buffer() [1.52%]
> Profile trace for function: rtalloc_ign_fib() [1.30%]
> Profile trace for function: mb_free_ext() [1.29%]
> Profile trace for function: arpresolve() [1.28%]
> Profile trace for function: bcopy() [1.22%]
> Profile trace for function: spinlock_exit() [1.06%]
> Profile trace for function: mb_ctor_pack() [1.04%]
> Profile trace for function: uma_zalloc_arg() [1.02%]
> Profile trace for function: spinlock_enter() [1.00%]
> Profile trace for function: bus_dmamap_load_mbuf_sg() [0.93%]
> Profile trace for function: critical_exit() [0.90%]
> Profile trace for function: igb_refresh_mbufs() [0.89%]
> Profile trace for function: acpi_cpu_c1() [0.89%]
> Profile trace for function: netisr_dispatch_src() [0.81%]
> Profile trace for function: uma_zfree_arg() [0.80%]
> Profile trace for function: lockstat_nsecs() [0.79%]
> Profile trace for function: ether_demux() [0.76%]
> Profile trace for function: in_lltable_lookup() [0.75%]
> Profile trace for function: ip_findroute() [0.66%]
> Profile trace for function: in_localip() [0.65%]
> Profile trace for function: _mtx_trylock_flags_() [0.57%]
> Profile trace for function: _bus_dmamap_load_buffer() [0.54%]
>
> Regards,
>
> Olivier
>
--
Ermal
More information about the svn-src-all
mailing list