From nobody Fri May 29 19:51:11 2026 X-Original-To: bugs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4gRv9q5wz2z6g238 for ; Fri, 29 May 2026 19:51:11 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R13" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4gRv9q2qM4z3wpS for ; Fri, 29 May 2026 19:51:11 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1780084271; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=bhIl1oFeQN8LlmmZ6bCBoGragPvoy2ntLKNcxJANCAs=; b=AjtXQe4+72VeGa9dgcZot0VfCg+mOKrSn8xJTt69nUVsRiU/TDFC/lhIzdpQZ/BmXcIRqC eBX3IjErRwxVcDIZjRAhIJ+GKc7TeMFA020TdddEQLyUGXfQTLmC2Qpcc3SDPTSrZZveX+ ds8sBe3LaFoK1PwDzMnbpMpQy7YV2lm4aA84CLfatBRf/vuo/mUIsUo9pO0FGbpCOT6lH0 V3BoExlYPjcOlIj8ypZEAasZw2BhQNMlBMw0T/CbrpwZLRnjI6EnpWjE0+2Sqz/o/w0uxT oRH1mMAVdBFCMxkdPVCr9c+mg7V1lFgA+GSDZB1siVZGvDx/eJPMggV37XlCFQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1780084271; a=rsa-sha256; cv=none; b=sqRT6/5f13/fRdkvK9dCcJ0pXAc3J7LFNeOzw6tJP+/JPkdoxig9xhw8AxbOo2BQxYDWJy Lz3n+saiowlWlzxPV3QfbANVWEI2o0caskxUB2YZAIuh0tekKMR/t4/ffK2VST16UMARi2 i7eeuDyQ9PxOmfVX8qMpWlFwYHNLfzpYPl9rSd+YZyhMcdKyiG1XofrjWjamypLAB4SmYl /rH/4Pm6z/uT9rDDbuUxU2SHPpRVMASzfMMT6EA+ZtMmrklqzZWZn0N96hf81BuNPtopg9 5rTW0/bm1+9RfCdF2WrGOiQyIxH9h+RKiUtgQeIgMcmMejwnPFGsj7X+o0EAJQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1780084271; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=bhIl1oFeQN8LlmmZ6bCBoGragPvoy2ntLKNcxJANCAs=; b=CN4TXWIVTrYA3pz09wW2zVJXkynEsEWwofCFdFMjpdBHE0lyccH8EPfzNa8PJHZue4RvDN nWVliWuRaWyG5DGEI9heStfKMT6nM3QTDevcyx4g2VMvAkvAIqybFrOe/Vnx6PO1DSEZDK ga/x03SupvORV+GFhszKXXQSTOpSBpxAnFZIAkQ4ULvMmGsq3M6yrDqUZn22JPMiUPVWpb ozm+44Q/Ro9A/cpNnvqIS0PDANIQ9iitZ6noUmpcQP6mdo8/UNHF6k7Yii34GZqMLYwVMX QWJHycAk6gX3dRbm8A4exdDKPe13Lwvure+V4PII489OwOt7I8ZPiqOTS/kWEg== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4gRv9q2QtfznZT for ; Fri, 29 May 2026 19:51:11 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 64TJpBW6056481 for ; Fri, 29 May 2026 19:51:11 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 64TJpBBA056480 for bugs@FreeBSD.org; Fri, 29 May 2026 19:51:11 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: bugs@FreeBSD.org Subject: [Bug 295707] aio_write: O_APPEND write ordering guarantee is not enforced Date: Fri, 29 May 2026 19:51:11 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 15.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: i.maximets@ovn.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform op_sys bug_status bug_severity priority component assigned_to reporter attachments.mimetype attachments.created Message-ID: Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="UTF-8" X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Bug reports List-Archive: https://lists.freebsd.org/archives/freebsd-bugs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-bugs@FreeBSD.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D295707 Bug ID: 295707 Summary: aio_write: O_APPEND write ordering guarantee is not enforced Product: Base System Version: 15.0-STABLE Hardware: Any OS: Any Status: New Severity: Affects Some People Priority: --- Component: kern Assignee: bugs@FreeBSD.org Reporter: i.maximets@ovn.org Attachment #271331 text/plain mime type: Created attachment 271331 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=3D271331&action= =3Dedit reproducer The man page [1] says: If O_APPEND is set for iocb->aio_fildes, write operations append to the file in the same order as the calls were made. [1] https://man.freebsd.org/cgi/man.cgi?query=3Daio_write Open vSwitch is using aio for logging, and we see a fairly frequent log reordering or even interleaving in our tests on FreeBSD. The reason seems to be that the kernel doesn't actually enforce the ordering for O_APPEND on the same file descriptor. It appears that kernel workers just pick up new requests whenever they can and so the writes end up out of order on systems with more than one core. AFAICT, POSIX technically has an exemption for the ordering rule for multiprocessor systems. However, this is not mentioned in the man page and the spirit of the exemption seems to actually be an exceptional case and not a general rule for how things should work. And aio in general would not be very useful if we had to wait for every request to be completed before submitting a new one. Attached a relatively simple reproducer program that mimics the usage pattern we have in OVS. It makes 50K writes with numbered lines with at most 256 requests in-flight at the same time. A ring buffer is used to track the requests. On EAGAIN - waits for one request to be done and tries again. At the end checks the file for the order of the written lines and the correctness of the written data. This test always passes on Linux, which has the same ordering claim in their man page, but different implementation, of course. On FreeBSD the test fails in our CI with ~25% of rows getting reordered: $ clang -o aio-append aio-append.c $ ./aio-append REORDERED at line 2: expected seq 1, got 2 REORDERED at line 5: expected seq 5, got 1 REORDERED at line 6: expected seq 2, got 5 REORDERED at line 11: expected seq 10, got 11 REORDERED at line 12: expected seq 12, got 10 REORDERED at line 13: expected seq 11, got 12 REORDERED at line 27: expected seq 26, got 27 REORDERED at line 28: expected seq 28, got 29 REORDERED at line 31: expected seq 32, got 26 REORDERED at line 32: expected seq 27, got 28 50000 lines, 13445 reordered, 0 corrupted While, I guess, that can be fixed by updating the docs while still being sort of POSIX compliant, would be nice to actually have kernel enforcing the currently documented behavior. WDYT? --=20 You are receiving this mail because: You are the assignee for the bug.=