From nobody Wed Jan 31 18:19:21 2024 X-Original-To: freebsd-threads@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TQ9L05PM2z59168 for ; Wed, 31 Jan 2024 18:19:36 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-ua1-f42.google.com (mail-ua1-f42.google.com [209.85.222.42]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TQ9Kz6cvWz4Pq4; Wed, 31 Jan 2024 18:19:35 +0000 (UTC) (envelope-from asomers@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=freebsd.org (policy=none); spf=pass (mx1.freebsd.org: domain of asomers@gmail.com designates 209.85.222.42 as permitted sender) smtp.mailfrom=asomers@gmail.com Received: by mail-ua1-f42.google.com with SMTP id a1e0cc1a2514c-7d2e16b552dso71622241.1; Wed, 31 Jan 2024 10:19:35 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706725174; x=1707329974; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=axxvuR4Pvkpi3mt+yELmqOi/IJhLsMW1kgpQ8LVknPo=; b=kWrUbJD9c52A1lrmJyNhFX2ScoNuF54VUJS9ArON/NASlRLeYI7l7YXF6JvC8Klq4I e8HVcxnlcmwTjvpIboLLmqHiIPtBF0JwfXIjFU76cX1LjkTjxS6VcCWPvcBELDxz47yo LW16RxfaXGQobf0UmFE1v9G8q+QiC88wmNG7SVm9G8mYQfgrdVyMPeEnXCq3jWNxOk/E penKynbr4frHXH0eeVetMEKCQvO4/B6kQANmXgfYmGNBB6ppsS/+gbyW9zXdGJRI2EPH Gg68f06LE4XwZLvtdKvjyBUtrgzeUcsLMkS8z+igiDTkgNtbGpvs0OvPH1cNrvgutBG0 zcgg== X-Gm-Message-State: AOJu0YxUzPPRqEHELYgAJeeDc2R3yz2EvS2nrF/z/jTH5EBdWEeRbH0q UW3t372leS+6c2LLsU5HhQOu38OILgzkfu2+buFBAST2KprdUFHH52HVI/XfUMDXSFGB+41o/sk 0ZCiQmsh1TNRi8KH8DsTXmGHsevo= X-Google-Smtp-Source: AGHT+IGJu9UiLcvUIVf0mwxDTXYnrNZu27ioihKMiV+uuYoWA9sKM6iHZ3q60ER3WY60hUUiW3Mbr84j+eGHrXSbEco= X-Received: by 2002:a05:6122:380d:b0:4b7:a77b:299 with SMTP id em13-20020a056122380d00b004b7a77b0299mr1941401vkb.16.1706725174217; Wed, 31 Jan 2024 10:19:34 -0800 (PST) List-Id: Threading List-Archive: https://lists.freebsd.org/archives/freebsd-threads List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-threads@freebsd.org X-BeenThere: freebsd-threads@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Alan Somers Date: Wed, 31 Jan 2024 11:19:21 -0700 Message-ID: Subject: Re: aio_read2() and aio_write2() To: =?UTF-8?Q?Vin=C3=ADcius_dos_Santos_Oliveira?= Cc: Konstantin Belousov , freebsd-threads@freebsd.org, Konstantin Belousov Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: / X-Spamd-Result: default: False [-0.92 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-0.999]; NEURAL_SPAM_SHORT(0.98)[0.978]; FORGED_SENDER(0.30)[asomers@freebsd.org,asomers@gmail.com]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17:c]; MIME_GOOD(-0.10)[text/plain]; DMARC_POLICY_SOFTFAIL(0.10)[freebsd.org : SPF not aligned (relaxed), No valid DKIM,none]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_TO(0.00)[gmail.com]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; FREEMAIL_CC(0.00)[gmail.com,freebsd.org]; TO_DN_SOME(0.00)[]; MIME_TRACE(0.00)[0:+]; RCPT_COUNT_THREE(0.00)[4]; RCVD_COUNT_ONE(0.00)[1]; FREEMAIL_ENVFROM(0.00)[gmail.com]; MLMMJ_DEST(0.00)[freebsd-threads@freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; FROM_NEQ_ENVFROM(0.00)[asomers@freebsd.org,asomers@gmail.com]; MISSING_XM_UA(0.00)[]; R_DKIM_NA(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[209.85.222.42:from]; TAGGED_RCPT(0.00)[]; FREEFALL_USER(0.00)[asomers]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; RCVD_IN_DNSWL_NONE(0.00)[209.85.222.42:from] X-Rspamd-Queue-Id: 4TQ9Kz6cvWz4Pq4 On Sun, Jan 14, 2024 at 12:07=E2=80=AFPM Vin=C3=ADcius dos Santos Oliveira wrote: > > Em dom., 14 de jan. de 2024 =C3=A0s 15:23, Alan Somers > escreveu: > > I think you're using the term "green threading" too broadly. Golang > > uses green threads, but Rust does not. The difference is that in Rust > > with async/await, the task-switching boundaries are explicit (the > > .await syntax). So Rust uses explicit concurrency, not green > > threading. I can't speak to the other languages you mention. > > Still, we might have async IO if the implementation permits. > > > You propose an extension that would essentially create asynchronous > > (and racy) versions of read, write, readv, and writev . But what > > about copy_file_range and fspacectl? Or for that matter all the > > dozens of control-path syscalls like open, stat, chmod, and truncate? > > They would block the thread, obviously. Look, I've been playing with > async IO for most of my career. I'm not asking for exoteric APIs. I > just want a non-blocking version for read(). What's so hard about > that? From what I understand from FreeBSD source code, I can already > "unofficially" do that (offset is ignored if the concrete type is not > a real file). Oh, are you not actually concerned about real files? aio_read and aio_write already have special handling for sockets. > > Very very few OSes actually implement async versions for anything > beyond the typical read()/write(). Even open() could block. For > anything beyond read()/write(), you just create a thread and live with > that. From a userspace perspective, it's expected that filesystem > operations such as file-move, directory-listing, etc will block the > thread. It's already expected. However you don't expect that for the > basic read()/write(). > > Again: Linux and Windows already allow that and it works fine on them. > > And again: I ask why FreeBSD is special here. I've been answering your > questions, but you've been avoiding my question every time. Why is > FreeBSD special here? Linux and Windows work just fine with this > design. Why suddenly does it become special for FreeBSD? It's the very > same application. The only sense in which FreeBSD is "special" is that we're better at finding the best solutions, rather than the quickest and hackiest. That's why we have kqueue instead of epoll, and ifconfig instead of ifconfig/iwconfig/wpa_supplicant/ip . > > > This flag that you propose is not a panacea that will eliminate all > > blocking file operations. There will still be a need for things that > > block. Rust (via the Tokio library) still uses a thread pool for such > > things. It even uses the thread pool for the equivalent of read() and > > write() (but not pread and pwrite). > > Nothing new here. I use thread pools to perform DNS queries. I allow > my user to create threads to perform blocking filesystem operations > (move, directory listing, etc). I know what I'm asking for: a read() > that won't block. I'm not asking for a competitor to io_uring. I'm > just asking for a read() that will never block my thread. > > > My point is that if you want fully asynchronous file I/O that never > > blocks you can't achieve that by adding one additional flag to POSIX > > AIO. > > It's just a read() that won't block the thread. Easy. > > Do you have concrete points for the design? What does it need to > change in the design so it becomes acceptable to you? What are the > criterias? If the implementation fulfills all these points, will it be > acceptable for you? I would like to see a design that: * Is extensible to most file system and networking syscalls, even if it doesn't include them right now. At a minimum, it should be able to include fspacectl, copy_file_range, truncate, and posix_fallocate. Probably open too. * Is reviewed by kib and Thomas Munro. * Has completion notification delivered by kqueue. * Is race-resistant. > > > Instead, all operations would > > either specify the offset (as with pwrite, pread) or operate only at > > EoF as if O_APPEND were used. > > I strongly disagree here. Async APIs should just achieve the same > semantics one *already* has when it creates threads and performs > blocking calls. Do *not* create new semantics. The initial patch > follows this principle. Your proposal does not. Shared state between asynchronous tasks is fundamentally racy if unsynchronized. And if synchronized, it fundamentally imposes a performance cost. I even recall reading a paper demonstrating that the need to assign file descriptors sequentially necessarily created shared state between threads. The authors were able to improve the performance of open() by assign file descriptors using some kind of thread-local data structure instead, and that way open() millions of files per second. That's what a good asynchronous API looks like: it's resistant to races without requiring extra synchronization. > > > -- > Vin=C3=ADcius dos Santos Oliveira > https://vinipsmaker.github.io/