svn commit: r360859 - projects/nfs-over-tls/sys/rpc

[Rick Macklem]

>Rick Macklem wrote:
>>John Baldwin wrote:
>>>On 5/9/20 5:17 PM, Rick Macklem wrote:
>>>> Author: rmacklem
>>>> Date: Sun May 10 00:17:39 2020
>>>> New Revision: 360859
>>>> URL: https://svnweb.freebsd.org/changeset/base/360859
>>>>
>>>> Log:
>>>>   Add some very basic handling of TLS_GET_RECORD control mbufs.
>>>>
>>>>   For now, it just throws away any that are non-application data.
>>>>   In the future, this will need to change, but not until TLS1.3, I think?
>>>
>>>Ideally you'd keep an nfsd thread in userland that you could pass
>>>these records onto.  One possible option is the thread just keeps
>>>calling SSL_read() but you do create a new flag on the socket buffer
>>>that causes soreceive() to only pass non-application data datagrams
>>>to userland reads() and have the in-kernel read requests block if they
>>>see a non-application data record as the next record until the user
>>>thread wakes up and reads it (or EAGAIN or whatever you need it to
>>>do).
Actually, what might work for the krpc code is a new MSG_TLSAPPDATA
flag for soreceive_generic(), which says "if the record is not application
data, return an error". (Sort of the opposite of what you said above, but
would perform the same thing.)
This could be used for the krpc soreceive() calls, so that the non-application
data record remains on the socket's receive buffer.

Then the krpc could do the upcall when the error is returned by soreceive()
and the userland daemon could do an SSL_read() with
SSL_MODE_AUTO_RETRY turned off. If I understand the man page, that will
make SSL_read() process the non-application data record but return with an
error of SSL_ERROR_WANT_READ without taking application data off the
socket's receive buffer queue.
--> If this all works (?), then the krpc can just go on and soreceive() the next
       application data record after the upcall returns.

Worth a try anyhow, I think? rick

>>Well, I currently have daemons (rpctlssd and rpctlscd) that just wait for
>>upcalls from the kernel and do the SSL stuff (mainly the handshake right now).
>(You can guess from the names which one is RPC client vs server.;-)
>I can easily do an upcall for a non-application data record if/when I need to do so.
>(The upcalls are done via Sun RPC using an AF_LOCAL socket, similar to what
> the gssd does.)
>
>For me, the mystery is what to do with it once the daemon gets it.
>From what you said, I'll need to "trick" SSL_read into reading it.
>Maybe I can push it back on the socket buffer receive queue in the kernel
>and then the daemon can do a SSL_read() to read it off the socket and handle
>it?
Oh, and one more little challenge...
When I played around with the daemons using TLS1.3 (before there was a ktls
rx I could use), I would run into early data that would be handled by SSL_read()
done in userland by the daemon.
However, I couldn't find a way to tell it to not wait to read any application data.
I recall trying an SSL_read() for 0 bytes and it didn't like it.
I might be stuck having the daemon do an SSL_read() for one application data
record and then it can pass that data back down into the kernel to be prepended
on the queue of received application data.

>(I wouldn't want to MSG_PEEK for every record, since these will be rare.)
>I also do already have code that blocks kernel reception when the upcall
>to do the handshake is done, so the same could be used in this case.
>
>There is the slight trick that the client krpc code is in a socket upcall that can't sleep,
>so I'll have to hand it off to some other thread that can sleep when I need to do it.
>
>Thanks for the hints, rick
rick
--
John Baldwin