svn commit: r360859 - projects/nfs-over-tls/sys/rpc

[Benjamin Kaduk]

On Mon, May 11, 2020 at 9:03 PM Rick Macklem <rmacklem at uoguelph.ca> wrote:

> >Rick Macklem wrote:
> >>John Baldwin wrote:
> >>>On 5/9/20 5:17 PM, Rick Macklem wrote:
> >>>> Author: rmacklem
> >>>> Date: Sun May 10 00:17:39 2020
> >>>> New Revision: 360859
> >>>> URL: https://svnweb.freebsd.org/changeset/base/360859
> >>>>
> >>>> Log:
> >>>>   Add some very basic handling of TLS_GET_RECORD control mbufs.
> >>>>
> >>>>   For now, it just throws away any that are non-application data.
> >>>>   In the future, this will need to change, but not until TLS1.3, I
> think?
> >>>
> >>>Ideally you'd keep an nfsd thread in userland that you could pass
> >>>these records onto.  One possible option is the thread just keeps
> >>>calling SSL_read() but you do create a new flag on the socket buffer
> >>>that causes soreceive() to only pass non-application data datagrams
> >>>to userland reads() and have the in-kernel read requests block if they
> >>>see a non-application data record as the next record until the user
> >>>thread wakes up and reads it (or EAGAIN or whatever you need it to
> >>>do).
>

You can avoid having to play games with putting stuff back on the socket
receive buffer by using a custom BIO implementation in userspace that knows
how to inject the received message.


> Actually, what might work for the krpc code is a new MSG_TLSAPPDATA
> flag for soreceive_generic(), which says "if the record is not application
> data, return an error". (Sort of the opposite of what you said above, but
> would perform the same thing.)
> This could be used for the krpc soreceive() calls, so that the
> non-application
> data record remains on the socket's receive buffer.
>
> Then the krpc could do the upcall when the error is returned by soreceive()
> and the userland daemon could do an SSL_read() with
> SSL_MODE_AUTO_RETRY turned off. If I understand the man page, that will
> make SSL_read() process the non-application data record but return with an
> error of SSL_ERROR_WANT_READ without taking application data off the
> socket's receive buffer queue.
>

The typical way to consume non-application-data records without hanging
trying
to read any application data is to do a zero-length read.  This still gets
far enough
into the state machine machinery to do the job before checking that the
length
is nonzero.


> --> If this all works (?), then the krpc can just go on and soreceive()
> the next
>        application data record after the upcall returns.
>
> Worth a try anyhow, I think? rick
>
> >>Well, I currently have daemons (rpctlssd and rpctlscd) that just wait for
> >>upcalls from the kernel and do the SSL stuff (mainly the handshake right
> now).
> >(You can guess from the names which one is RPC client vs server.;-)
> >I can easily do an upcall for a non-application data record if/when I
> need to do so.
> >(The upcalls are done via Sun RPC using an AF_LOCAL socket, similar to
> what
> > the gssd does.)
> >
> >For me, the mystery is what to do with it once the daemon gets it.
> >From what you said, I'll need to "trick" SSL_read into reading it.
> >Maybe I can push it back on the socket buffer receive queue in the kernel
> >and then the daemon can do a SSL_read() to read it off the socket and
> handle
> >it?
> Oh, and one more little challenge...
> When I played around with the daemons using TLS1.3 (before there was a ktls
> rx I could use), I would run into early data that would be handled by
> SSL_read()
> done in userland by the daemon.
> However, I couldn't find a way to tell it to not wait to read any
> application data.
> I recall trying an SSL_read() for 0 bytes and it didn't like it.
>

In the early-data case things are more complicated.  Calling regular
SSL_read() will
drive the handshake to completion, and there's a separate function to call
to just
try to read early data.  (You could also configure things to fully deny
early data which
would probably be easier.)

-Ben


> I might be stuck having the daemon do an SSL_read() for one application
> data
> record and then it can pass that data back down into the kernel to be
> prepended
> on the queue of received application data.
>
> >(I wouldn't want to MSG_PEEK for every record, since these will be rare.)
> >I also do already have code that blocks kernel reception when the upcall
> >to do the handshake is done, so the same could be used in this case.
> >
> >There is the slight trick that the client krpc code is in a socket upcall
> that can't sleep,
> >so I'll have to hand it off to some other thread that can sleep when I
> need to do it.
> >
> >Thanks for the hints, rick
> rick
> --
> John Baldwin
> _______________________________________________
> svn-src-projects at freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/svn-src-projects
> To unsubscribe, send any mail to "svn-src-projects-unsubscribe at freebsd.org
> "
>