From nobody Thu Apr 06 14:26:04 2023 X-Original-To: questions@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PskMD3P4xz44QSC for ; Thu, 6 Apr 2023 14:26:16 +0000 (UTC) (envelope-from pete@nomadlogic.org) Received: from mail.nomadlogic.org (mail.nomadlogic.org [66.165.241.226]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "mail.nomadlogic.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PskM945rDz3GW4 for ; Thu, 6 Apr 2023 14:26:13 +0000 (UTC) (envelope-from pete@nomadlogic.org) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=nomadlogic.org header.s=04242021 header.b=mEFDIjTQ; spf=pass (mx1.freebsd.org: domain of pete@nomadlogic.org designates 66.165.241.226 as permitted sender) smtp.mailfrom=pete@nomadlogic.org; dmarc=pass (policy=quarantine) header.from=nomadlogic.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nomadlogic.org; s=04242021; t=1680791165; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=oB7+P4b1wgLqoxo1rowvG+cUScvYaMW44M/dLfPwhqM=; b=mEFDIjTQ0FlEBmq3YdHjOZgwnPxLabUCbomRAnqNvkslUaEnUqKoCsWCAK3aJIQ4HgbAN7 ql8ILnda6jCH99OBkwIw32rU/DbTbGviAZOnt3y6msBUyfYx4lAlXiPpADrhj+gScTBG2B jN6ZHbOeCINoE3KWs6uBZEcImkKMYes= Received: from shelly.nomadlogic.org (66-165-241-229.static.hvvc.us [66.165.241.229]) by mail.nomadlogic.org (OpenSMTPD) with ESMTPSA id 1ef0face (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Thu, 6 Apr 2023 14:26:05 +0000 (UTC) Date: Thu, 6 Apr 2023 14:26:04 +0000 From: Pete Wright To: Pete Cc: Matthias Apitz , Christian Weisgerber , questions@freebsd.org Subject: Re: Clogged pipe? Message-ID: <20230406142604.dug4liknps2zuvvz@shelly.nomadlogic.org> References: <4b563b2a-2c26-8c84-f8db-51e4f17dcaf6@slagle.net> List-Id: User questions List-Archive: https://lists.freebsd.org/archives/freebsd-questions List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-questions@freebsd.org X-BeenThere: freebsd-questions@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4b563b2a-2c26-8c84-f8db-51e4f17dcaf6@slagle.net> X-Spamd-Result: default: False [-3.00 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; SUBJECT_ENDS_QUESTION(1.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.998]; DMARC_POLICY_ALLOW(-0.50)[nomadlogic.org,quarantine]; R_DKIM_ALLOW(-0.20)[nomadlogic.org:s=04242021]; R_SPF_ALLOW(-0.20)[+mx]; MIME_GOOD(-0.10)[text/plain]; MLMMJ_DEST(0.00)[questions@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:29802, ipnet:66.165.240.0/22, country:US]; MIME_TRACE(0.00)[0:+]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MID_RHS_MATCH_FROMTLD(0.00)[]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; DKIM_TRACE(0.00)[nomadlogic.org:+]; RCPT_COUNT_THREE(0.00)[4]; RCVD_COUNT_TWO(0.00)[2]; TO_DN_SOME(0.00)[]; RCVD_TLS_ALL(0.00)[] X-Rspamd-Queue-Id: 4PskM945rDz3GW4 X-Spamd-Bar: -- X-ThisMailContainsUnwantedMimeParts: N On Tue, Apr 04, 2023 at 12:25:36AM -0700, Pete wrote: > Matthias Apitz wrote on 4/3/23 22:36: > > Well, I was just showing a simplified example to illustrate the problem; > what I was really trying to script is this: > > /usr/bin/tail -f -n+1 /var/log/exim/main-$(/bin/date -u '+%Y%m%d').log | > /bin/cat -n | > /usr/bin/sed -Eu 's/^[[:blank:]]*([[:digit:]]*)[[:blank:]]+/\1 /' > > which prefixes line numbers as it watches today's exim log file scroll > along. So, in this case the -f flag is needed on the tail command. > > What actually fixes the buffering problem on FreeBSD (it already works fine > on Linux) is to add the -u flag to the cat command. > > Jan Schaumann had a great blog post on pipes buffer sizes a while back that may be interesting: https://www.netmeister.org/blog/ipcbufs.html It dives into the internals and does a pretty great break down of how various OS's handle buffering. -pete -- Pete Wright pete@nomadlogic.org