git: 1ecbc1d8e9d3 - main - cxgbe tom: Don't queue AIO requests on listen sockets.

[Alan Somers]

On Wed, Sep 15, 2021 at 9:21 AM John Baldwin <jhb at freebsd.org> wrote:

> On 9/14/21 1:53 PM, Alan Somers wrote:
> > On Tue, Sep 14, 2021 at 2:46 PM John Baldwin <jhb at freebsd.org> wrote:
> >
> >> The branch main has been updated by jhb:
> >>
> >> URL:
> >>
> https://cgit.FreeBSD.org/src/commit/?id=1ecbc1d8e9d3fbcd8e68fc68f0a32944a12ddb1e
> >>
> >> commit 1ecbc1d8e9d3fbcd8e68fc68f0a32944a12ddb1e
> >> Author:     John Baldwin <jhb at FreeBSD.org>
> >> AuthorDate: 2021-09-14 20:46:14 +0000
> >> Commit:     John Baldwin <jhb at FreeBSD.org>
> >> CommitDate: 2021-09-14 20:46:14 +0000
> >>
> >>      cxgbe tom: Don't queue AIO requests on listen sockets.
> >>
> >>      This is similar to the fixes in 141fe2dceeae.  One difference is
> that
> >>      TOE sockets do not change states (listen vs non-listen) once
> created,
> >>      so no lock is needed for SOLISTENING().
> >>
> >>      Sponsored by:   Chelsio Communications
> >>
> >
> > I've always wondered: what's the point to using AIO with sockets?  Can't
> > everything socket-related be done better with non-blocking read/write and
> > kqueue?
>
> Zero-copy operation with TOE is why TOE uses AIO.  Zero-copy of user
> buffers
> can't really work with the non-AIO APIs because the user buffer is free to
> be reused immediately after write(2) (and on the read side you don't know
> the buffer in advance to allow the NIC to write directly into the use
> buffer).
>
> In theory we could support zero-copy using mb_ext_pgs for aio_write() for
> the non-TOE case similar to what sendfile() does.
>
> --
> John Baldwin
>

Interesting.  Do you know of any common applications that include this
optimization?  I've been working on the AIO ecosystem for Rust.  It would
be good to ensure that this use case works, especially if zero-copy ever
works for non-TOE.