Server sporadically sending unexpected RST to Client

[Seth Jeacopello]

Thanks for the quick reply Andre; we have some new information.

First I took some time to review some of the tcpdumps per your
recommendation and have not found /any/ reuse (with most dumps spanning
approx. a one hour time frame and the problem occurring toward the end of
the time frame).

The client system is another FreeBSD system (we are unsure of the version at
this time).

You may be correct about the syncache simply showing the symptoms; as we dug
deeper we began looking at changes in netisr, in particular the direct
dispatch policy modifications.  We've run some tests over the weekend and
found something that seems to work for us.

We've found that moving from 'Always Direct' to 'Hybrid' mode seems to
resolve the issue for us without any noticed consequences (setting
net.isr.direct_force=0).  Can anyone comment on this setting and let us know
of any downsides or problems that may occur running in this mode?

We believe that this problem is also only isolated to one of our Server
platforms (testing on our other platform is still on-going, though initial
results look good).

Both platforms are Intel based (current generation vs. last generation) with
various differences, though the one that may be most relative is the change
of the on-board NIC from being 'em' based to 'igb' based (that is the
systems with the issue all have 'em' based NICs vs. 'igb' of the newer
systems).  This could be red-herring as well, though I feel it's probably a
good idea to include as much information as possible when troubleshooting.

Thank you for all of your help and I look forward to hearing any further
thoughts on this issue.

--Seth