From nobody Wed May 19 04:17:02 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id D23845CA0D8 for ; Wed, 19 May 2021 04:17:10 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FlKMV3St3z3HR7; Wed, 19 May 2021 04:17:10 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.16.1/8.16.1) with ESMTPS id 14J4H3vX092398 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Wed, 19 May 2021 07:17:06 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 14J4H3vX092398 Received: (from kostik@localhost) by tom.home (8.16.1/8.16.1/Submit) id 14J4H3vc092397; Wed, 19 May 2021 07:17:03 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 19 May 2021 07:17:02 +0300 From: Konstantin Belousov To: Alan Somers Cc: Mark Johnston , FreeBSD Hackers Subject: Re: The pagedaemon evicts ARC before scanning the inactive page list Message-ID: References: List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on tom.home X-Rspamd-Queue-Id: 4FlKMV3St3z3HR7 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] On Tue, May 18, 2021 at 09:55:25PM -0600, Alan Somers wrote: > On Tue, May 18, 2021 at 9:25 PM Konstantin Belousov > > Is your machine ZFS-only? If yes, then typical source of inactive memory > > can be of two kinds: > > > > No, there is also FUSE. But there is typically < 1GB of Buf memory, so I > didn't mention it. As Mark mentioned, buffers use page cache as second-level cache. More precisely, there is relatively limited number of buffers in the system, which are just headers to describe a set of pages. When a buffer is recycled, its pages are put on inactive queue. This is why I asked is your machine ZFS-only or not, because io on bufcache-using filesystems typically add to the inactive queue. > > > > - anonymous memory that apps allocate with facilities like malloc(3). > > If inactive is shrinkable then it is probably not, because dirty pages > > from anon objects must go through laundry->swap route to get evicted, > > and you did not mentioned swapping > > > > No, there's no appreciable amount of swapping going on. Nor is the laundry > list typically more than a few hundred MB. > > > > - double-copy pages cached in v_objects of ZFS vnodes, clean or dirty. > > If unmapped, these are mostly a waste. Even if mapped, the source > > of truth for data is ARC, AFAIU, so they can be dropped as well, since > > inactive state means that its content is not hot. > > > > So if a process mmap()'s a file on ZFS and reads from it but never writes > to it, will those pages show up as inactive? It depends on workload, and it does not matter much if the pages are clean or dirty. Right after mapping or under intense access pattern, they sit on the active list. If not touched long enough, or cycled through the buffer cache for io (but ZFS pages not go through buffer cache), they are moved to inactive. > > > > > > You can try to inspect the most outstanding objects adding to the > > inactive queue with 'vmobject -o' to see where the most of inactive pages > > come from. > > > > Wow, that did it! About 99% of the inactive pages come from just a few > vnodes which are used by the FUSE servers. But I also see a few large > entries like > 1105308 333933 771375 1 0 WB df > what does that signify? These are anonymous memory. > > > > > > If indeed they are double-copy, then perhaps ZFS can react even to the > > current primitive vm_lowmem signal somewhat different. First, it could > > do the pass over its vnodes and > > - free clean unmapped pages > > - if some targets are not met after that, laundry dirty pages, > > then return to freeing clean unmapped pages > > all that before ever touching its cache (ARC). > >