Seeing EINVAL from writev on 8.0 to a non-blocking socket even though the data seems to hit the wire

[Richard Sharpe]

On Thu, May 2, 2013 at 7:52 AM, Eric van Gyzen <eric at vangyzen.net> wrote:
> On 05/02/2013 08:48, Richard Sharpe wrote:
>> On Wed, May 1, 2013 at 9:34 PM, Alfred Perlstein <bright at mu.org> wrote:
>>> On 5/1/13 8:03 PM, Richard Sharpe wrote:
>>>> Hi folks,
>>>>
>>>> I am checking to see if there are any known bugs with respect to this
>>>> in FreeBSD 8.0.
>>>>
>>>> Situation is that Samba 3.6.6 uses writev to a non-blocking socket to
>>>> get the SMB2 requests on the wire.
>>>>
>>>> Intermittently, we see the writev return EINVAL even though the data
>>>> has gotten on the wire. This I have verified by grabbing a capture and
>>>> comparing the SMB Sequence number in the last outgoing packet on the
>>>> wire vs the in-memory contents when we get EINVAL.
>>>>
>>>> Sometimes it occurs on a four-element IOVEC, sometimes we get EAGAIN
>>>> on the four-element IOVEC and then we get EINVAL when retrying on a
>>>> smaller IOVEC.
>>>>
>>>> Where should I look to check if there is some path where this might be
>>>> happening? Is this even the correct mailing list?
>>>>
>>> What does the iovec look like when you get EINVAL? Can you sanity check
>>> it? Is there anything special about it? (zero length vecs?)
>>>
>>> I think there are a few "maxvals" that if overrun cause EINVAL to be
>>> returned. example is if your iovec is somehow huge or has many, many
>>> elements.
>> Can anyone tell me the call graph down to the TCP code?
>>
>
> writev kern/sys_generic.c
> kern_writev
> dofilewrite
> fo_write in sys/file.h
> soo_write in kern/sys_socket.c
> sosend in kern/uipc_socket.c
> sosend_generic
> tcp_usr_send in netinet/tcp_usrreq.c

Thanks for that. The only places it seems that EINVAL could be coming
from are the last two, and they seem to be valid error cases and one
of those seem like it could never happen in this path (kern_writev
sends down flags set to zero).

Since it has the feel of a race, I need to find out of we have a race
with signal handlers in some way and whether or not we are doing
something weird with control messages or something on the socket.

-- 
Regards,
Richard Sharpe
(何以解憂？唯有杜康。--曹操)