From nobody Wed Sep 13 08:46:22 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RlvFQ36bvz4thL6 for ; Wed, 13 Sep 2023 08:46:34 +0000 (UTC) (envelope-from theraven@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RlvFQ2gT4z3cd1; Wed, 13 Sep 2023 08:46:34 +0000 (UTC) (envelope-from theraven@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1694594794; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kWHV2pDdCDq1WWBha9AXloR9JMiEtubA8JItTiyYPqo=; b=BZcUnKt7KzwaoDsEsKezARmnnJFo6vXo4C4J8eNkU+b3MBTaVjkju3UbBe+ILcbHGmynK6 qgmxmEqo1EnsmE/vjfBoVav5X9KYMNR6gQ5lhKyWnTPbrbIJSs1vIl76b72dIaeJWmQgqa E4bGxJ7+WWLrcA2vw9VRoVJhWvQM28I5cpO4b1zweujLnORWQ+2okDeJXRVLEmjp4e1X1G fM1aZPU57JKvGcNHf9+QW131ukcZSnOmQd7xi97DiphNiSKz/UanbzTNpQV5lSotCLUZGS NmQNfFsHjmpaX9AlaQ6qsq6rxt0XkmiMaCJBapV/QaLFecrmAvlcQ7Wc6WH1ZA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1694594794; a=rsa-sha256; cv=none; b=uL2VEdyL3vZiC/lethJvOUkpLYdpvyOwflhjNXP0v0v9jGVy6anW4v+rwUcJVn12ZjgXo3 VAVtZD9foEz8ODokdumXNiVpSNESt3KZblTnFjH3uy3IpCbT1I4cQ0co+jU+1QsPjHGYuQ uhRVS5gaCZaO2uE6J5r8crHRUVTgDXEF+YuWGvcNPQ2Hjni6gD+kMxxqHWR7sefQyiaqSE SGLwqA/oRzk1dUH55xcvVaIDKWfBhcP3890NCLxWNK/mLtjKyIYIx1a+VGHf8XKHT2CIpe zeFU269nmIjPNpc72uUJX7BVU3QYM5ftysWbY22T/wzuNZgD1Dnh4yqL1i+AGg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1694594794; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kWHV2pDdCDq1WWBha9AXloR9JMiEtubA8JItTiyYPqo=; b=K40hb5RjBTWptbHxsPUFJgp/Pxdk1jSgaBLBujffYOM2pHYgToJLUnCvs+JqPghGwK/LsZ TPWYYnaHvx30HQwLFx31Gy5WsKbIWsnVhfzUXhpodVcNZTTqIVMSNWYdfXHOExkwk9XomJ B9b4XeH6WRi0QdEyS3DSVZzdhwxvGyEYNgdCUCnYaOObcFYV4aIpWzJfgNh1jiwahRj4fs jM8lP6mP6xeklLtXjO538/8rDN/cXmF7mW6+Tv3AjLp0ftlHyQ/sJsfHaTGwO7a9TTFO+Z yx54zcTafJrzUu0lIECpauXtSRK20Yr9rnJset0mJIAq9hb+Dcd4MiJjHqXzOA== Received: from smtp.theravensnest.org (smtp.theravensnest.org [45.77.103.195]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: theraven) by smtp.freebsd.org (Postfix) with ESMTPSA id 4RlvFQ1bYVzrkS; Wed, 13 Sep 2023 08:46:34 +0000 (UTC) (envelope-from theraven@FreeBSD.org) Received: from smtpclient.apple (host81-156-206-121.range81-156.btcentralplus.com [81.156.206.121]) by smtp.theravensnest.org (Postfix) with ESMTPSA id 651239C8E; Wed, 13 Sep 2023 09:46:33 +0100 (BST) Content-Type: text/plain; charset=utf-8 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Subject: Re: Continually count the number of open files From: David Chisnall In-Reply-To: <592123F4-E610-446E-82B4-ACC519C0BA3E@iitbombay.org> Date: Wed, 13 Sep 2023 09:46:22 +0100 Cc: Graham Perrin , FreeBSD CURRENT Content-Transfer-Encoding: quoted-printable Message-Id: <1D86A8FB-ACC6-427E-ABB0-2E1A5989170E@FreeBSD.org> References: <291ad2de-ba0e-4bdf-786a-19614eacec49@gmail.com> <592123F4-E610-446E-82B4-ACC519C0BA3E@iitbombay.org> To: Bakul Shah X-Mailer: Apple Mail (2.3731.700.6) On 12 Sep 2023, at 17:19, Bakul Shah wrote: >=20 > FreeBSD > should add inotify. inotify is also probably not the right thing. If someone is interested = in adding this, Apple=E2=80=99s fsevents API is a better inspiration. = It is carefully designed to ensure that the things monitoring for events = can=E2=80=99t ever block filesystem operations from making progress. I = think there=E2=80=99s a nice design possible with a bloom filter in the = kernel of events that ensures that monitors may get spurious events but = don=E2=80=99t miss out on anything. =20 On macOS because files have a stronger idea of which directory they live = in and so it=E2=80=99s easier to have an API that notifies for changes = to files in a directory. inotify monitors individual files but loses = notifications if you write through the wrong hard link to a file (hard = link foo from a/foo to b/foo, use inotify to watch a, write though = b/foo, observe no notification). I think the right kernel API would = walk the directory and add the vnodes to a bloom filter and trigger a = notification on a match in the filter. You=E2=80=99d then have = occasional spurious notifications but you=E2=80=99d have something that = could be monitored via kqueue and could be made to not block anything = else in the kernel. If anyone is interested in improving the current kqueue code here: = there=E2=80=99s currently no mechanism for tracking when the last = writable file descriptor for a file has been closed, which is useful for = consuming files that are dropped via sftp or similar. NOTE_CLOSED_WRITE = is hard to use without race conditions and tells you only that *a* file = descriptor with write permission has closed, not that the last one has = closed. I=E2=80=99m currently resorting to a process that runs as root = that uses libprocstat to walk the entire list of open file descriptors = and report that they=E2=80=99re closed, which is incredibly suboptimal. David