From nobody Sat Jan 29 07:12:30 2022 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id BCBB819701AE; Sat, 29 Jan 2022 07:12:30 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Jm5B64QQdz3QBr; Sat, 29 Jan 2022 07:12:30 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1643440350; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=1oVU8xBHaW6jPmmCkOg88YiAZH7puozZsjODXZ558kQ=; b=J8rf8c97I90hE5OEXltCDI3KgM6EEBKobFqwkuFQFAY+EEimsyN24nUjFUjhQBRjNIhBc4 GJj252sSZs3shXUulkNoRc6IWX0qOu2Cqh7Wdl43+yyRXlOQtl9gDEX9qrJmnCcgNC6FyK YdyjOuAjnvwRB5ot9kXj8S8x0Y/QXs37RmChiP0pEhaDfA8hnV2+g/N6ric42G9Cf00guL FZvuRISCHDozvBH4/mb5+IlTTYTUyXt0awuLmeKKgOx9tPeww7Sp6T8nlKWZMKAEEHwmLu +urisA1eZ32X1Nv32gUovX4kTgIJuGqAY6X3mzejr+G3/ZnDYI3iHQHkupBw3w== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 601C9114FE; Sat, 29 Jan 2022 07:12:30 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 20T7CUW2024235; Sat, 29 Jan 2022 07:12:30 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 20T7CU0O024234; Sat, 29 Jan 2022 07:12:30 GMT (envelope-from git) Date: Sat, 29 Jan 2022 07:12:30 GMT Message-Id: <202201290712.20T7CU0O024234@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Wolfram Schneider Subject: git: 21cc4bee6962 - main - sync improvements from concatdb.sh List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: wosch X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 21cc4bee6962d3d56e5f7b66cd614991f739e7ed Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1643440350; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=1oVU8xBHaW6jPmmCkOg88YiAZH7puozZsjODXZ558kQ=; b=cxF8+QlExmDz7/5xKs+Zelra+JjpVND8/K7o9knbVeq0zLrdS77aZxNtlPMxVTv+1qxXCe qPaJus4B/3evTBkwZGg4+e10gWKrKpZkZdg3nwkELroF1IZqKUwq8W45Z7NqQL/7odjkfA l4mnHmA3cgnLphuY9xqDhpFaQGkm/I/1m7qbMIsSeMwzAiVp9KbQHhLG4+QFJvmNp5n2MS 8n4mDSdtQRH0tkqj9XiCtnakzSkAj/teOeqLdeKskyYE7y7esZEC6FWW+ixmoskWfHGPwN I5VWrFp95//tKlYi4EtehFpf0GoxVDxDJT9cfwC8xaAcjHuaO2oyDAa6pq0zvw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1643440350; a=rsa-sha256; cv=none; b=fXRJt3JxYPOIuNl4EmKIi/TGK2NZa/u4VbjmrlQUGtnVs0RfRHqQXmGbBpRQNK3HKBV9Lj skCHmIYiPIpzikyY4nz2kuRI/VPZtKvSAoJ9vgQ43AGgb+dPyvI33m2mADgpX84/ZQJCTI 5uT5HFcRSDq653TqPh3cWWcEtWy/YZts3bd+/6u9gaeGJ55v9O+IEuQD7GP50XMLMbfiX0 weyE/dOWweo9ufLhzPJPiuDWUMh8xZSca7PwJw84iA+bWsA2gaQ5AmS+Qa48owU1oC06j5 t1ROOjhEw5JFlVyFFs4fpH/OhLUDfokYW2NRzf9FEEmNrPUlbwmUZ3bFRine9g== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by wosch: URL: https://cgit.FreeBSD.org/src/commit/?id=21cc4bee6962d3d56e5f7b66cd614991f739e7ed commit 21cc4bee6962d3d56e5f7b66cd614991f739e7ed Author: Wolfram Schneider AuthorDate: 2022-01-29 07:10:18 +0000 Commit: Wolfram Schneider CommitDate: 2022-01-29 07:12:09 +0000 sync improvements from concatdb.sh - stop on first error - improve awk script: print the last two characters for bigram - not the second word - remove unnecessary checks - use mktemp - refactor --- usr.bin/locate/locate/mklocatedb.sh | 55 ++++++++++++++----------------------- 1 file changed, 20 insertions(+), 35 deletions(-) diff --git a/usr.bin/locate/locate/mklocatedb.sh b/usr.bin/locate/locate/mklocatedb.sh index e838c9795820..fd4a1ff7f03b 100644 --- a/usr.bin/locate/locate/mklocatedb.sh +++ b/usr.bin/locate/locate/mklocatedb.sh @@ -2,7 +2,7 @@ # # SPDX-License-Identifier: BSD-2-Clause-FreeBSD # -# Copyright (c) September 1995 Wolfram Schneider . Berlin. +# Copyright (c) September 1995-2022 Wolfram Schneider # All rights reserved. # # Redistribution and use in source and binary forms, with or without @@ -32,43 +32,32 @@ # # $FreeBSD$ +# stop on first error +set -e +set -o pipefail + # The directory containing locate subprograms : ${LIBEXECDIR:=/usr/libexec}; export LIBEXECDIR +: ${TMPDIR:=/var/tmp}; export TMPDIR PATH=$LIBEXECDIR:/bin:/usr/bin:$PATH; export PATH -umask 077 # protect temp files - -: ${TMPDIR:=/tmp}; export TMPDIR -test -d "$TMPDIR" || TMPDIR=/tmp -if ! TMPDIR=`mktemp -d $TMPDIR/mklocateXXXXXXXXXX`; then - exit 1 -fi - - # utilities to built locate database : ${bigram:=locate.bigram} : ${code:=locate.code} : ${sort:=sort} +sort_opt="-u -T $TMPDIR -S 20%" -sortopt="-u -T $TMPDIR -S 20%" -sortcmd=$sort - - -bigrams=$TMPDIR/_mklocatedb$$.bigrams -filelist=$TMPDIR/_mklocatedb$$.list - -trap 'rm -f $bigrams $filelist; rmdir $TMPDIR' 0 1 2 3 5 10 15 +bigrams=$(mktemp -t mklocatedb.bigrams) +filelist=$(mktemp -t mklocatedb.filelist) +trap 'rm -f $bigrams $filelist' 0 1 2 3 5 10 15 # Input already sorted if [ X"$1" = "X-presort" ]; then shift; - # create an empty file - true > $bigrams - # Locate database bootstrapping # 1. first build a temp database without bigram compression # 2. create the bigram from the temp database @@ -76,19 +65,15 @@ if [ X"$1" = "X-presort" ]; then # # This scheme avoid large temporary files in /tmp - $code $bigrams > $filelist || exit 1 - locate -d $filelist / | $bigram | $sort -nr | head -128 | - awk '{if (/^[ ]*[0-9]+[ ]+..$/) {printf("%s",$2)} else {exit 1}}' > $bigrams || exit 1 - locate -d $filelist / | $code $bigrams || exit 1 - exit - + $code $bigrams > $filelist + locate -d $filelist / | $bigram | $sort -nr | \ + awk 'NR <= 128 && /^[ \t]*[1-9][0-9]*[ \t]+..$/ { printf("%s", substr($0, length($0)-1, 2)) }' > $bigrams + locate -d $filelist / | $code $bigrams else - if $sortcmd $sortopt > $filelist; then - $bigram < $filelist | $sort -nr | - awk '{if (/^[ ]*[0-9]+[ ]+..$/) {printf("%s",$2)} else {exit 1}}' > $bigrams || exit 1 - $code $bigrams < $filelist || exit 1 - else - echo "`basename $0`: cannot build locate database" >&2 - exit 1 - fi + $sort $sort_opt > $filelist + $bigram < $filelist | $sort -nr | \ + awk 'NR <= 128 && /^[ \t]*[1-9][0-9]*[ \t]+..$/ { printf("%s", substr($0, length($0)-1, 2)) }' > $bigrams + $code $bigrams < $filelist fi + +#EOF