how to use the ktls

[John Baldwin]

On 1/12/20 8:23 PM, Benjamin Kaduk wrote:
> On Thu, Jan 09, 2020 at 10:53:38PM +0000, Rick Macklem wrote:
>> John Baldwin wrote:
>>> On 1/7/20 3:02 PM, Rick Macklem wrote:
>>>> Hi,
>>>>
>>>> Now that I've completed NFSv4.2 I'm on to the next project, which is making NFS
>>>> work over TLS.
>>>> Of course, I know absolutely nothing about TLS, which will make this an interesting
>>>> exercise for me.
>>>> I did find simple server code in the OpenSSL doc. which at least gives me a starting
>>>> point for the initialization stuff.
>>>> As I understand it, this initialization must be done in userspace?
>>>>
>>>> Then somehow, the ktls takes over and does the encryption of the
>>>> data being sent on the socket via sosend_generic(). Does that sound right?
>>>>
>>>> So, how does the kernel know the stuff that the initialization phase (handshake)
>>>> figures out, or is it magic I don't have to worry about?
>>>>
>>>> Don't waste much time replying to this. A few quick hints will keep me going for
>>>> now. (From what I've seen sofar, this TLS stuff isn't simple. And I thought Kerberos
>>>> was a pain.;-)
>>>>
>>>> Thanks in advance for any hints, rick
>>>
>>> Hmmm, this might be a fair bit of work indeed.
>> If it was easy,  it wouldn't be fun;-) FreeBSD13 is a ways off and if it doesn't make that, oh well..
>>
>>> Right now KTLS only works for transmit (though I have some WIP for receive).
>> Hopefully your WIP will make progress someday, or I might be able to work on it.
>>
>>> KTLS does assumes that the initial handshake and key negotiation is handled by
>>> OpenSSL.  OpenSSL uses custom setockopt() calls to tell the kernel which
>>> session keys to use.
>> Yea, I figured I'd need a daemon like the gssd for this. The krpc makes it a little
>> more fun, since it handles TCP connections in the kernel.
>>
>>> I think what you would want to do is use something like OpenSSL_connect() in
>>> userspace, and then check to see if KTLS "worked".
>> Thanks (and for the code below). I found the simple server code in the OpenSSL doc,
>> but the client code gets a web page and is quite involved.
>>
>>> If it did, you can tell
>>> the kernel it can write to the socket directly, otherwise you will have to
>>> bounce data back out to userspace to run it through SSL_write() and have
>>> userspace do SSL_read() and then feed data into the kernel.
>> I don't think bouncing the data up/down to/from userland would work well.
>> I'd say "if it can't be done in the kernel, too bad". The above could be used for
>> a NULL RPC to see it is working, for the client.
> 
> So you're saying that we'd only support rpc-over-tls as an NFS client and
> not as a server, at least until the WIP for ktls read appears?

To be clear, I have KTLS RX working with TOE right now.  I have a design in my
head for KTLS RX that would use software and co-processor engines via OCF such
as aesni(4) and ccr(4) that I hope to implement in the next few months, so KTLS
RX isn't too far off.  OpenSSL already supports KTLS RX on Linux and the FreeBSD
patches I already have use the same API.  (Each received TLS frame is read via
recvmsg() with the TLS header fields in a cmsg.)

-- 
John Baldwin