Getting started with ktls

[Alan Somers]

I'm trying to make ktls work with "zfs send/recv" to substantially reduce
the CPU utilization of applications like zrepl.  But I have a few questions:

* ktls(4)'s "Transmit" section says "Once TLS transmit is enabled by a
successful set of the TCP_TXTLS_ENABLE socket option", but the "Supported
Libraries" section says "Applications using a supported library should
generally work with ktls without any changes".  These sentences seem to be
contradictory.  I think it means that the TCP_TXTLS_ENABLE option is
necessary, but OpenSSL sets it automatically?

* When using OpenSSL, the library will automatically call setsockopt(_,
TCP_TXTLS_ENABLE).  But it swallows the error, if any.  How is an
application to tell if ktls is enabled on a particular socket or OpenSSL
session?

* From experiment, I can see that OpenSSL attempts to set
TCP_TXTLS_ENABLE.  But it doesn't try to set TCP_RXTLS_ENABLE.  Why not?
>From reading ktls_start and ossl_statem_server_post_work, it looks like
maybe a single socket cannot have ktls enabled for both sending and
receiving at the same time.  Is that true?

Based on the man page and rmacklem's previous mailing list posts, I think
this should be workable with minor modifications to the kernel and libzfs.
I just need to figure out how to use ktls first.

-Alan