Server sporadically sending unexpected RST to Client

[Seth Jeacopello]

Interestingly the patch you provided is almost identical to what we put in
place on the system to trace the issue earlier.  Without having merged the
patch yet, let me give you what we do have from that run:

Aug  6 11:15:27 kernel: Entering syncache_add 

Aug  6 11:15:27 kernel: Entering syncache_expand 

Aug  6 11:15:27 kernel: TCP: [10.174.50.35]:52872 to [10.174.80.242]:389
tcpflags 0x18<PUSH,ACK>; syncache_expand: Just before syncache_socket() 

Aug  6 11:15:27 kernel: Entering syncache_socket 

Aug  6 11:15:27 kernel: Entering syncache_expand 

Aug  6 11:15:27 kernel: TCP: [10.174.50.35]:52872 to [10.174.80.242]:389
tcpflags 0x10<ACK>; syncache_expand: Just before syncache_socket() 

Aug  6 11:15:27 kernel: Entering syncache_socket 

Aug  6 11:15:27 kernel: TCP: [10.174.50.35]:52872 to [10.174.80.242]:389;
syncache_socket: Socket create failed in_pcbconnect: 48 

Aug  6 11:15:27 kernel: TCP: [10.174.50.35]:52872 to [10.174.80.242]:389
tcpflags 0x10<ACK>; tcp_input: Listen socket: Socket allocation failed due
to limits or memory shortage, sending RST


Also, I wanted to note again that the tcpdumps we've grabbed are showing
that the packets are going across the wire in order, however, debug logging
of the issue on the Server side has shown they appear to be processed out of
order (with the PSH, ACK being processed before the final ACK of the
handshake).


--Seth