From nobody Fri Apr 29 18:08:22 2022 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id AF12B1ABE0CE for ; Fri, 29 Apr 2022 18:08:26 +0000 (UTC) (envelope-from pete@nomadlogic.org) Received: from mail.nomadlogic.org (mail.nomadlogic.org [66.165.241.226]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "mail.nomadlogic.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4KqgTP4gw6z4mg3 for ; Fri, 29 Apr 2022 18:08:25 +0000 (UTC) (envelope-from pete@nomadlogic.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nomadlogic.org; s=04242021; t=1651255703; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1C5cQBZYHdGBVAQ3u3s7z9YW3IJcKBXnzXYHALpdK6c=; b=qTxk/FLHo8hPDKMJX3AnB9KhDfjf6z3B1mzMpk3DlDo+dX3BucIIZYQestCPw/6/GySksD UP9+j+3/eWzH+8f0pw6G7E74JpWjn2NKJ1WwrJ7NK1ioXv47JY5rAb2rPffou0ICnwoRnE /SExDBfT7O6/CoC1Qx07hXcGalEVE1A= Received: from [192.168.1.160] (cpe-24-24-168-214.socal.res.rr.com [24.24.168.214]) by mail.nomadlogic.org (OpenSMTPD) with ESMTPSA id a6d6fd56 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Fri, 29 Apr 2022 18:08:22 +0000 (UTC) Message-ID: Date: Fri, 29 Apr 2022 11:08:22 -0700 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:91.0) Gecko/20100101 Thunderbird/91.8.1 Subject: Re: Chasing OOM Issues - good sysctl metrics to use? Content-Language: en-US From: Pete Wright To: Mark Millard Cc: freebsd-current References: <83A713B9-A973-4C97-ACD6-830DF6A50B76.ref@yahoo.com> <83A713B9-A973-4C97-ACD6-830DF6A50B76@yahoo.com> <94B2E2FD-2371-4FEA-8E01-F37103F63CC0@yahoo.com> <0fcb5a4a-5517-e57b-2b69-4f3b3b10589a@nomadlogic.org> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4KqgTP4gw6z4mg3 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=nomadlogic.org header.s=04242021 header.b="qTxk/FLH"; dmarc=pass (policy=quarantine) header.from=nomadlogic.org; spf=pass (mx1.freebsd.org: domain of pete@nomadlogic.org designates 66.165.241.226 as permitted sender) smtp.mailfrom=pete@nomadlogic.org X-Spamd-Result: default: False [-2.74 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[nomadlogic.org:s=04242021]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[nomadlogic.org:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[nomadlogic.org,quarantine]; NEURAL_HAM_SHORT(-0.74)[-0.742]; MLMMJ_DEST(0.00)[freebsd-current]; FREEMAIL_TO(0.00)[yahoo.com]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; SUBJECT_ENDS_QUESTION(1.00)[]; ASN(0.00)[asn:29802, ipnet:66.165.240.0/22, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[] X-ThisMailContainsUnwantedMimeParts: N On 4/23/22 19:20, Pete Wright wrote: > >> The developers handbook has a section debugging deadlocks that he >> referenced in a response to another report (on freebsd-hackers). >> >> https://docs.freebsd.org/en/books/developers-handbook/kerneldebug/#kerneldebug-deadlocks >> > > d'oh - thanks for the correction! > > -pete > > hello, i just wanted to provide an update on this issue.  so the good news is that by removing the file backed swap the deadlocks have indeed gone away!  thanks for sorting me out on that front Mark! i still am seeing a memory leak with either firefox or chrome (maybe both where they create a voltron of memory leaks?).  this morning firefox and chrome had been killed when i first logged in. fortunately the system has remained responsive for several hours which was not the case previously. when looking at my metrics i see vm.domain.0.stats.inactive take a nose dive from around 9GB to 0 over the course of 1min.  the timing seems to align with around the time when firefox crashed, and is proceeded by a large spike in vm.domain.0.stats.active from ~1GB to 7GB 40mins before the apps crashed.  after the binaries were killed memory metrics seem to have recovered (laundry size grew, and inactive size grew by several gigs for example). maybe i'll have to gather data and post it online for anyone who would be interested in seeing this in graph form.  although, frankly i feel like it's a browser problem which i can work around by running them in jails with resource limits in place via rctl. -pete -- Pete Wright pete@nomadlogic.org @nomadlogicLA