Interrupt performance

[Bruce Evans]

On Tue, 1 Feb 2011, Slawa Olhovchenkov wrote:

> I do some more test and build kernel with KTR.
> Now I don't think that inetrrupt overhead on FreeBSD weight: I try
> polling and don't see any difference.
>
> I see many reported by netperf send errors. I found this
> http://docs.freebsd.org/cgi/mid.cgi?E1Aice9-0002by-00.
>
> After insert into src/nettest_bsd.c usleep(1000) if ENOBUF I see 53%
> idle and ./loop 2000000000 "Elapsed 15188006 us" -- this near to linux
> (Elapsed 14107670 us).

This partly works around the problem that it is impossible to select()
on the ENOBUFS condition in FreeBSD at least, and thus impossible to
write ttcp or nettest correctly.  The userland sender either has to
sleep for a while it gets an ENOBUFS error, and thus let the hardware
sender go idle in the interval between the condition becoming clear
and the sleep finishing, or it has to retry immediately and thus consume
100% CPU getting ENOBUFS errors until the condition clears.  I use
HZ = 100.  Thus usleep(1000) would actually sleep for an average of 15000
us, and the system would be idle (doing nothing) for about 10 times
as long as with HZ = 1000.  I uses an old version of ttcp which tries
to sleep for 18000 us.  This ensures that the the sleep is too long
even with HZ = 1000 (except I changed 1 line in the old ttcp to either
not sleep at all or to try to sleep for only 1000 us).  Not sleeping
at all uses 100% CPU, but since I mostly use this for testing the
maximum packet rate I don't care much about that unless the CPU being
used by ttcp interferes with kernel and/or hardware activity.

Another reply said that Linux blocks on ENOBUFS instead of returning
it.  That seems better, provided it doesn't block in the O_NOBLOCK case.
This should involve select() working so that you can avoid the block
even in the !O_NONBLOCK case.  Correct versions of ttcp and maybe
nettest can then be written very easily -- at least ttcp would prefer
to just block in sendto().

Bruce