Packet loss every 30.999 seconds

[Bruce Evans]

On Mon, 17 Dec 2007, Mark Fullmer wrote:

> Thanks.  Have a kernel building now.  It takes about a day of uptime after 
> reboot before I'll see the problem.

Yes run "find / >/dev/null" to see the problem if it is the syncer one.

At least the syscall latency problem does seem to be this.  Under ~5.2,
with the above find and also "while :; do sync; done" (to give latency
spike more often), your program (with some fflush(stdout)'s and args
1 7700) gives:

% 1197976029041677 12696 0
% 1197976033196396 9761 4154719
% 1197976034060031 13360 863635
% 1197976039080632 13749 5020601
% 1197976043195594 8536 4114962
% 1197976044100601 13505 905007
% 1197976049121870 14562 5021269
% 1197976052195631 8192 3073761
% 1197976054141545 14024 1945914
% 1197976059162357 14623 5020812
% 1197976063195735 7830 4033378
% 1197976064182564 14618 986829
% 1197976069202982 14823 5020418
% 1197976074223722 15350 5020740
% 1197976079244311 15726 5020589
% 1197976084264690 15893 5020379
% 1197976089289409 15058 5024719
% 1197976094315433 16209 5026024
% 1197976095197277 8015 881844
% 1197976099335529 16092 4138252
% 1197976104356513 16863 5020984
% 1197976109376236 16373 5019723
% 1197976114396803 16727 5020567
% 1197976119416822 16533 5020019
% 1197976124437790 17288 5020968
% 1197976126200637 10060 1762847
% 1197976127198459 7839 997822
% 1197976129457321 16606 2258862
% 1197976134477582 16654 5020261

This clearly shows the spike every 5 seconds, and the latency creeping
up as vfs.numvnodes increases.  It started at about 20000 and ended at
about 64000.

The syncer won't be fixed soon, so the fix for dropped packets requires
figuring out why the syncer affects networking.

Bruce