FreeBSD 11.1 contigfree performance issue

[Kononov, Oleksandr]

I am using FreeBSD 11.1 -RELEASE-amd64 running on a single 32 core CPU and am having issues with contigmalloc performance.
Timing the function using rdtsc shows that it uses up on average about 10 million cycles on that function along.
Using the same code, FreeBSD and timing method I ran it on anther machine on two CPU's with a total of 32 cores.
This gave about 12 thousand cycles on that function.

Digging through the source code (on the single CPU) I found that

smp_targeted_tlb_shootdown function in /usr/src/sys/x86/x86/mp_x86.c
cause the majority of performance hit due to some cores remaining in a paused state longer after the
interrupt was send to them.


I attached a sample code and Makefile in this email.

Steps to recreate (and show rdtsc cycles):

$ make
$ kldload ./test.ko
$ dmesg

If anyone has any idea what is the cause of this issue, it would be greatly appreciated.
--------------------------------------------------------------
Intel Research and Development Ireland Limited
Registered in Ireland
Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
Registered Number: 308263


This e-mail and any attachments may contain confidential material for the sole
use of the intended recipient(s). Any review or distribution by others is
strictly prohibited. If you are not the intended recipient, please contact the
sender and delete all copies.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Makefile
Type: application/octet-stream
Size: 317 bytes
Desc: Makefile
URL: <http://lists.freebsd.org/pipermail/freebsd-drivers/attachments/20180327/84c01b66/attachment.obj>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: test.c
URL: <http://lists.freebsd.org/pipermail/freebsd-drivers/attachments/20180327/84c01b66/attachment.c>