Puzzling performance

[John Baldwin]

On Monday, August 02, 2010 6:13:50 pm Guy Helmer wrote:
> On a FreeBSD 7.1 SCHED_ULE kernel, I have a large number of files opened and
> mmapped (with MAP_NOSYNC option) for shared-memory communication between
> processes.  Normally, memcpy() copies data into these shared-memory buffers
> in a reasonable amount of time closely related to the size of the copy
> (roughly 10us per 10KB).  However, due to performance issues I've found that
> sometimes a memcpy() takes an abnormally long time (10ms for 40KB, and I
> suspect longer times occurring when I have not had monitoring enabled).  The
> system doesn't seem to be in memory overcommit -- there is just a minor
> amount of swap in use, and I've not seen page-ins or page-outs while
> watching systat or vmstat.
> 
> Since I'm using MAP_NOSYNC, I would not expect the pager to flush dirty
> pages to disk and cause add delays.  Any ideas where to look?  Might it help
> to pin threads to CPUs in case a thread is getting moved to a different
> core?

Pinning might help yes.  You might also want to ensure there aren't any 
interrupts on that CPU.  Currently there isn't a good way to figure that out 
short of kgdb though. :(

-- 
John Baldwin