Contigfree takes too much time that cuases PCIe driver unload to take up to 30 minutes

[Steevan Rodrigues]

Hi ,

After some investigation I came across following email thread:
http://freebsd.1045724.x6.nabble.com/FreeBSD-11-1-contigfree-performance-issue-td6247284.html

Now I see that the issue seen by me is same as the one described in the
above email thread. We found that our driver too takes too much time during
configfree and contigmalloc calls. This happens only in server with xeon
dual CPU with 10 core each ( total 20 cores / 40 threads).
This issue is not seen in server with Xeon Dual CPU with 6 core each (
total 12 cores / 24 threads)

Any suggestions about how to overcome this problem with contigmalloc and
configfree ?
Thanks
Steevan


On Fri, Sep 21, 2018 at 9:32 AM Steevan Rodrigues <steevanxperia at gmail.com>
wrote:

> Hello Folks,
>
> We have  a PCI express card for data processing to achieve 25 to 30 Gbps .
> Recently we have been facing a issue in one of the server at customer
> site.
>
> System information:
> Supermicro motherboard-X11DPH-TQ motherboard
> hw.model: Intel(R) Xeon(R) Gold 5115 CPU @ 2.40GHz  ( Dual CPU with total
> 20 cores)
> 16 GB RAM
>
> In this system the same PCIe card works fine in RHEL 7.5 and the driver
> load and unload also works fine.  Our driver usually takes about 20 seconds
> to load and approx 20 seconds to unload.
>
> However, on this particular system when FreeBSD ( tried with 11.1 and
> 11.2  Release) is used it takes about 2 to 4 minutes to load and about 8 to
> 30 minutes to unload.
> During unload it looks like the system freezes completely and I can not
> run any commands to find out what is happening .
>
> The same driver works fine in our lab  servers Dell T620 ( Xeon 12 core
> CPUs) and desktops with FreeBSD 11.1 and 11.2 and 10.4 .
>
> Also I ran couple of Phoronix tests on this SuperMicro server. In one
> particular test I see that the time taken is too much.
> The test suite is OsBench. In this, the thread creation test shows average
> time taken to create threads is 9000 usec .  On the other hand we have
> servers and desktops in our lab and in that this same thread creation takes
> only 20 to 30 usec .
>
> Any pointers about what could be wrong with the system or our PCIe card
> driver  ?
>
> Thanks
> Steevan
>
>
>
>