From nobody Mon May 08 08:05:25 2023 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4QFDP14Knrz49YLM; Mon, 8 May 2023 08:05:25 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4QFDP13qsbz3j36; Mon, 8 May 2023 08:05:25 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1683533125; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=nyUwxcNBySNgsyYKTYpvi3ssb3mJn9hcWy9rjE0Tjg0=; b=YvxtnRQvHm4gIkqedxQUynWlttkESPHB2pSjj0QyVCD/FBsJyMXhN29Zvnv8OhFseNHfHS du2bfh07f++Lwt+QOyUSurkJmbEZm4+2sm7gvMyqp7/5XQMVVodq7itv4TRB5Awf81B1wA 6jIReZZArql4pPqfNVB1y/E7YozjN6nD3sde+3LP97HiNuak7Klv8QDmEI80JV/WEbYFWQ 4sqquMSbQdaMp9S7mHiLfiDEVMvXVYp2gwo0Px4BndVTkWzYHk9Ju5hc1X64BE2eBjDeTt 4ub75KdrVY/UoYRa3qcGa7XJiNFKS/PkSnH43eIkHjkbcFWseoif8r7m6x4Mew== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1683533125; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=nyUwxcNBySNgsyYKTYpvi3ssb3mJn9hcWy9rjE0Tjg0=; b=u49mg3VSYDnduhuZIVA5jf1zbFcTxgGPLEoiRqak6y6eIuXE1DoLVolw5+9JqZW8/IK9Jf T6A51SymXgjPrmGmhN2HnXhW/m9RQJIfOPOpB0g7KKJGScoX5+3X0v4dJQTfRGiV8O1Ln0 dEbmO6SINT9NaVv3/yjKr0uDjBT9SovCdvjWerg9TT2utD8WS74Aw/oJpb3lfR2Zr/Sd4n W/SL46A9EcvakRmRD9VQwEntuj85M8THhuWNu+HxBb3zmMsXtMcPdUAodxzzT8Uh0avPsz lhF2F7omx5JyBZGO63frqzd3eZNs278dUFZU59jLhxhZiJYuJpOYy+sn+fXqiQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1683533125; a=rsa-sha256; cv=none; b=ILTz6RzG1jpxsBtPxxXaSllku7fpEsS6TWCYwFFb2U9snFQYUhSJ6penGN5owHvC/aF3Be EVMFw54GwRvutKDEPuCnhbLo2pSi/wL8GuKJBz7bqoQAKhb4A0n/kAtwgE2DLlnb6s3SHO B3z1E/h0Wgq62C808Zw8+WTf3kZ8wRaI6/4tAvMuUvu/LIhrge6WyXYqn0OFf0ispQ0Lcm H2/hKOQPovz2qw9sgAoqsK/JivaSHR32+/iA60npjcj1vae9xQqrERJuvytnJwgCcX7xwy C+1ThbSqG0tDjskKGeY9N9al5xgmxTSOBhxKZKyvfgB+9eMFY+45IwaholcxIg== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4QFDP12n6fzslQ; Mon, 8 May 2023 08:05:25 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 34885Pa8017964; Mon, 8 May 2023 08:05:25 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 34885P8c017963; Mon, 8 May 2023 08:05:25 GMT (envelope-from git) Date: Mon, 8 May 2023 08:05:25 GMT Message-Id: <202305080805.34885P8c017963@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: =?utf-8?Q?Dag-Erling=20Sm=C3=B8rgrav?= Subject: git: 4849767cb16a - main - md5: Improve compatibility. List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: des X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 4849767cb16a4dbd4d1b923db25d34029c09e7b0 Auto-Submitted: auto-generated X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by des: URL: https://cgit.FreeBSD.org/src/commit/?id=4849767cb16a4dbd4d1b923db25d34029c09e7b0 commit 4849767cb16a4dbd4d1b923db25d34029c09e7b0 Author: Dag-Erling Smørgrav AuthorDate: 2023-05-08 06:56:09 +0000 Commit: Dag-Erling Smørgrav CommitDate: 2023-05-08 06:56:22 +0000 md5: Improve compatibility. * Overhaul the GNU compatibility mode to more closely emulate what the GNU tools do. * Add a Perl compatibility mode which emulates the shasum tool that ships with Perl. This is currently not installed. * Overhaul the tests. Sponsored by: Klara, Inc. Reviewed by: kevans Differential Revision: https://reviews.freebsd.org/D39446 --- ObsoleteFiles.inc | 73 ++++ sbin/md5/Makefile | 6 + sbin/md5/md5.1 | 313 +++++++++++---- sbin/md5/md5.c | 665 ++++++++++++++++++++++++-------- sbin/md5/tests/1.inp | 0 sbin/md5/tests/1.sha512-p.chk | 1 - sbin/md5/tests/1.sha512sum-p.chk | 1 - sbin/md5/tests/2.inp | 1 - sbin/md5/tests/2.sha512-p.chk | 1 - sbin/md5/tests/2.sha512sum-p.chk | 1 - sbin/md5/tests/3.inp | 1 - sbin/md5/tests/3.sha512-p.chk | 1 - sbin/md5/tests/3.sha512sum-p.chk | 1 - sbin/md5/tests/4.inp | 1 - sbin/md5/tests/4.sha512-p.chk | 1 - sbin/md5/tests/4.sha512sum-p.chk | 1 - sbin/md5/tests/5.inp | 1 - sbin/md5/tests/5.sha512-p.chk | 1 - sbin/md5/tests/5.sha512sum-p.chk | 1 - sbin/md5/tests/6.inp | 1 - sbin/md5/tests/6.sha512-p.chk | 1 - sbin/md5/tests/6.sha512sum-p.chk | 1 - sbin/md5/tests/7.inp | 1 - sbin/md5/tests/7.sha512-p.chk | 1 - sbin/md5/tests/7.sha512sum-p.chk | 1 - sbin/md5/tests/8.inp | 1 - sbin/md5/tests/8.sha512-p.chk | 1 - sbin/md5/tests/8.sha512sum-p.chk | 1 - sbin/md5/tests/Makefile | 35 -- sbin/md5/tests/bsd-c-test.SH | 23 -- sbin/md5/tests/bsd-p-test.SH | 24 -- sbin/md5/tests/bsd-s-test.SH | 30 -- sbin/md5/tests/coreutils-c-test.SH | 21 - sbin/md5/tests/md5.digest | 8 - sbin/md5/tests/md5_test.sh | 346 +++++++++++++++-- sbin/md5/tests/md5sum.digest | 8 - sbin/md5/tests/rmd160.digest | 8 - sbin/md5/tests/rmd160sum.digest | 8 - sbin/md5/tests/self-test.SH | 8 - sbin/md5/tests/self-test.md5.chk | 9 - sbin/md5/tests/self-test.rmd160.chk | 9 - sbin/md5/tests/self-test.sh_inp | 8 - sbin/md5/tests/self-test.sha1.chk | 9 - sbin/md5/tests/self-test.sha224.chk | 9 - sbin/md5/tests/self-test.sha256.chk | 9 - sbin/md5/tests/self-test.sha384.chk | 9 - sbin/md5/tests/self-test.sha512.chk | 9 - sbin/md5/tests/self-test.sha512t224.chk | 9 - sbin/md5/tests/self-test.sha512t256.chk | 9 - sbin/md5/tests/self-test.skein1024.chk | 9 - sbin/md5/tests/self-test.skein256.chk | 9 - sbin/md5/tests/self-test.skein512.chk | 9 - sbin/md5/tests/sha1.digest | 8 - sbin/md5/tests/sha1sum.digest | 8 - sbin/md5/tests/sha224.digest | 8 - sbin/md5/tests/sha224sum.digest | 8 - sbin/md5/tests/sha256.digest | 8 - sbin/md5/tests/sha256sum.digest | 8 - sbin/md5/tests/sha384.digest | 8 - sbin/md5/tests/sha384sum.digest | 8 - sbin/md5/tests/sha512.digest | 8 - sbin/md5/tests/sha512sum.digest | 8 - sbin/md5/tests/sha512t224.digest | 8 - sbin/md5/tests/sha512t224sum.digest | 8 - sbin/md5/tests/sha512t256.digest | 8 - sbin/md5/tests/sha512t256sum.digest | 8 - sbin/md5/tests/skein1024.digest | 8 - sbin/md5/tests/skein1024sum.digest | 8 - sbin/md5/tests/skein256.digest | 8 - sbin/md5/tests/skein256sum.digest | 8 - sbin/md5/tests/skein512.digest | 8 - sbin/md5/tests/skein512sum.digest | 8 - sbin/md5/tests/sum_a.in | 1 - sbin/md5/tests/sum_b.in | 1 - sbin/md5/tests/sum_c.in | 1 - sbin/md5/tests/sum_sums.digest | 3 - 76 files changed, 1134 insertions(+), 747 deletions(-) diff --git a/ObsoleteFiles.inc b/ObsoleteFiles.inc index ede2871455ae..97881f40ac59 100644 --- a/ObsoleteFiles.inc +++ b/ObsoleteFiles.inc @@ -52,6 +52,79 @@ # xargs -n1 | sort | uniq -d; # done +# 20230505: md5 tests are now self-contained +OLD_FILES+=usr/tests/sbin/md5/1.inp +OLD_FILES+=usr/tests/sbin/md5/1.sha512-p.chk +OLD_FILES+=usr/tests/sbin/md5/1.sha512sum-p.chk +OLD_FILES+=usr/tests/sbin/md5/2.inp +OLD_FILES+=usr/tests/sbin/md5/2.sha512-p.chk +OLD_FILES+=usr/tests/sbin/md5/2.sha512sum-p.chk +OLD_FILES+=usr/tests/sbin/md5/3.inp +OLD_FILES+=usr/tests/sbin/md5/3.sha512-p.chk +OLD_FILES+=usr/tests/sbin/md5/3.sha512sum-p.chk +OLD_FILES+=usr/tests/sbin/md5/4.inp +OLD_FILES+=usr/tests/sbin/md5/4.sha512-p.chk +OLD_FILES+=usr/tests/sbin/md5/4.sha512sum-p.chk +OLD_FILES+=usr/tests/sbin/md5/5.inp +OLD_FILES+=usr/tests/sbin/md5/5.sha512-p.chk +OLD_FILES+=usr/tests/sbin/md5/5.sha512sum-p.chk +OLD_FILES+=usr/tests/sbin/md5/6.inp +OLD_FILES+=usr/tests/sbin/md5/6.sha512-p.chk +OLD_FILES+=usr/tests/sbin/md5/6.sha512sum-p.chk +OLD_FILES+=usr/tests/sbin/md5/7.inp +OLD_FILES+=usr/tests/sbin/md5/7.sha512-p.chk +OLD_FILES+=usr/tests/sbin/md5/7.sha512sum-p.chk +OLD_FILES+=usr/tests/sbin/md5/8.inp +OLD_FILES+=usr/tests/sbin/md5/8.sha512-p.chk +OLD_FILES+=usr/tests/sbin/md5/8.sha512sum-p.chk +OLD_FILES+=usr/tests/sbin/md5/algorithms.txt +OLD_FILES+=usr/tests/sbin/md5/bsd-c-test +OLD_FILES+=usr/tests/sbin/md5/bsd-p-test +OLD_FILES+=usr/tests/sbin/md5/bsd-s-test +OLD_FILES+=usr/tests/sbin/md5/coreutils-c-test +OLD_FILES+=usr/tests/sbin/md5/md5.digest +OLD_FILES+=usr/tests/sbin/md5/md5sum.digest +OLD_FILES+=usr/tests/sbin/md5/rmd160.digest +OLD_FILES+=usr/tests/sbin/md5/rmd160sum.digest +OLD_FILES+=usr/tests/sbin/md5/self-test +OLD_FILES+=usr/tests/sbin/md5/self-test.md5.chk +OLD_FILES+=usr/tests/sbin/md5/self-test.rmd160.chk +OLD_FILES+=usr/tests/sbin/md5/self-test.sh_inp +OLD_FILES+=usr/tests/sbin/md5/self-test.sha1.chk +OLD_FILES+=usr/tests/sbin/md5/self-test.sha224.chk +OLD_FILES+=usr/tests/sbin/md5/self-test.sha256.chk +OLD_FILES+=usr/tests/sbin/md5/self-test.sha384.chk +OLD_FILES+=usr/tests/sbin/md5/self-test.sha512.chk +OLD_FILES+=usr/tests/sbin/md5/self-test.sha512t224.chk +OLD_FILES+=usr/tests/sbin/md5/self-test.sha512t256.chk +OLD_FILES+=usr/tests/sbin/md5/self-test.skein1024.chk +OLD_FILES+=usr/tests/sbin/md5/self-test.skein256.chk +OLD_FILES+=usr/tests/sbin/md5/self-test.skein512.chk +OLD_FILES+=usr/tests/sbin/md5/sha1.digest +OLD_FILES+=usr/tests/sbin/md5/sha1sum.digest +OLD_FILES+=usr/tests/sbin/md5/sha224.digest +OLD_FILES+=usr/tests/sbin/md5/sha224sum.digest +OLD_FILES+=usr/tests/sbin/md5/sha256.digest +OLD_FILES+=usr/tests/sbin/md5/sha256sum.digest +OLD_FILES+=usr/tests/sbin/md5/sha384.digest +OLD_FILES+=usr/tests/sbin/md5/sha384sum.digest +OLD_FILES+=usr/tests/sbin/md5/sha512.digest +OLD_FILES+=usr/tests/sbin/md5/sha512sum.digest +OLD_FILES+=usr/tests/sbin/md5/sha512t224.digest +OLD_FILES+=usr/tests/sbin/md5/sha512t224sum.digest +OLD_FILES+=usr/tests/sbin/md5/sha512t256.digest +OLD_FILES+=usr/tests/sbin/md5/sha512t256sum.digest +OLD_FILES+=usr/tests/sbin/md5/skein1024.digest +OLD_FILES+=usr/tests/sbin/md5/skein1024sum.digest +OLD_FILES+=usr/tests/sbin/md5/skein256.digest +OLD_FILES+=usr/tests/sbin/md5/skein256sum.digest +OLD_FILES+=usr/tests/sbin/md5/skein512.digest +OLD_FILES+=usr/tests/sbin/md5/skein512sum.digest +OLD_FILES+=usr/tests/sbin/md5/sum_a.in +OLD_FILES+=usr/tests/sbin/md5/sum_b.in +OLD_FILES+=usr/tests/sbin/md5/sum_c.in +OLD_FILES+=usr/tests/sbin/md5/sum_sums.digest + # 20230420: portsnap.8 removed OLD_FILES+=etc/portsnap.conf OLD_FILES+=usr/libexec/make_index diff --git a/sbin/md5/Makefile b/sbin/md5/Makefile index 6bda75437275..c9faec285aea 100644 --- a/sbin/md5/Makefile +++ b/sbin/md5/Makefile @@ -52,6 +52,12 @@ MLINKS= md5.1 md5sum.1 \ md5.1 skein1024.1 \ md5.1 skein1024sum.1 +# md5 can also emulate the shasum script that comes with Perl, except +# that, in bits input mode, it can only handle input lengths that are +# a multiple of 8 (see manual page). +#LINKS+= ${BINDIR}/md5 ${BINDIR}/shasum +#MLINKS+= md5.1 shasum.1 + LIBADD= md .ifndef(BOOTSTRAPPING) diff --git a/sbin/md5/md5.1 b/sbin/md5/md5.1 index ba654e131c3c..bd619587e7a9 100644 --- a/sbin/md5/md5.1 +++ b/sbin/md5/md5.1 @@ -1,5 +1,5 @@ .\" $FreeBSD$ -.Dd February 6, 2023 +.Dd April 12, 2023 .Dt MD5 1 .Os .Sh NAME @@ -8,7 +8,8 @@ .Nm rmd160 , skein256 , skein512 , skein1024 , .Nm md5sum , sha1sum , sha224sum , sha256sum , sha384sum , .Nm sha512sum , sha512t224sum , sha512t256sum , -.Nm rmd160sum , skein256sum , skein512sum , skein1024sum +.Nm rmd160sum , skein256sum , skein512sum , skein1024sum , +.Nm shasum .Nd calculate a message-digest fingerprint (checksum) for a file .Sh SYNOPSIS .Nm @@ -18,12 +19,40 @@ .Op Ar .Pp .Nm md5sum -.Op Fl pqrtx -.Op Fl c Ar file -.Op Fl s Ar string +.Op Fl bctwz +.Op Fl -binary +.Op Fl -check +.Op Fl -help +.Op Fl -ignore-missing +.Op Fl -quiet +.Op Fl -status +.Op Fl -strict +.Op Fl -tag +.Op Fl -text +.Op Fl -version +.Op Fl -warn +.Op Fl -zero .Op Ar .Pp (All other hashes have the same options and usage.) +.Pp +.Nm shasum +.Op Fl 0bchqstUvw +.Op Fl -01 +.Op Fl a | -algorithm Ar alg +.Op Fl -binary +.Op Fl -check +.Op Fl -help +.Op Fl -ignore-missing +.Op Fl -quiet +.Op Fl -status +.Op Fl -strict +.Op Fl -tag +.Op Fl -text +.Op Fl -UNIVERSAL +.Op Fl -version +.Op Fl -warn +.Op Ar .Sh DESCRIPTION The .Nm md5 , sha1 , sha224 , sha256 , sha384 , sha512 , sha512t224 , sha512t256 , @@ -36,15 +65,21 @@ output a or .Dq message digest of the input. +.Pp The .Nm md5sum , sha1sum , sha224sum , sha256sum , sha384sum , sha512sum , .Nm sha512t224sum , sha512t256sum , rmd160sum , skein256sum , skein512sum , and .Nm skein1024sum -utilities do the same, but default to the reversed format of -the -.Fl r -flag. +utilities do the same, but with command-line options and an output +format that match those of their similary named GNU utilities. +.Pp +The +.Nm shasum +utility does the same, but with command-line options and an output +format that match those of the similarly named utility that ships with +Perl. +.Pp It is conjectured that it is computationally infeasible to produce two messages having the same message digest, or to produce any message having a given prespecified target message digest. @@ -75,73 +110,171 @@ to 224 bits. .Pp It is recommended that all new applications use SHA-512 or SKEIN-512 instead of one of the other hash functions. -.Pp -The following options may be used in any combination and must -precede any files named on the command line. -The hexadecimal checksum of each file listed on the command line is printed -after the options are processed. +.Ss BSD OPTIONS +The following options are available in BSD mode, i.e. when the program +is invoked with a name that does not end in +.Dq sum : .Bl -tag -width indent -.It Fl b -Make the -.Nm -sum -programs separate hash and digest with a blank followed by an asterisk instead -of by 2 blank characters for full compatibility with the output generated by the -coreutils versions of these programs. -.It Fl c Ar string -If the program was called with a name that does not end in -.Nm sum , -compare the digest of the file against this string. +.It Fl c Ar string , Fl -check= Ns Ar string +Compare the digest of the file against this string. If combined with the .Fl q +or +.Fl -quiet option, the calculated digest is printed in addition to the exit status being set. .Pq Note that this option is not yet useful if multiple files are specified. -.It Fl c Ar file -If the program was called with a name that does end in -.Nm sum , -the file passed as argument must contain digest lines generated by the same -digest algorithm with or without the -.Fl r -option -.Pq i.e., in either classical BSD format or in GNU coreutils format . -A line with the file name followed by a colon -.Dq ":" -and either OK or FAILED is written for each well-formed line in the digest file. -If applicable, the number of failed comparisons and the number of lines that were -skipped since they were not well-formed are printed at the end. -The -.Fl q -option can be used to quiesce the output unless there are mismatched entries in -the digest. -.Pp -.It Fl s Ar string -Print a checksum of the given -.Ar string . -.It Fl p +.It Fl p , -passthrough Echo stdin to stdout and append the checksum to stdout. -.It Fl q +.It Fl q , -quiet Quiet mode \(em only the checksum is printed out. Overrides the .Fl r +or +.Fl -reverse option. -.It Fl r +.It Fl r , -reverse Reverses the format of the output. This helps with visual diffs. Does nothing when combined with the .Fl ptx options. -.It Fl t +.It Fl s Ar string , Fl -string= Ns Ar string +Print a checksum of the given +.Ar string . +.It Fl t , Fl -time-trial Run a built-in time trial. For the .Nm -sum versions, this is a nop for compatibility with coreutils. -.It Fl x +.It Fl x , Fl -self-test Run a built-in test script. .El +.Ss GNU OPTIONS +The following options are available in GNU mode, i.e. when the program +is invoked with a name that ends in +.Dq sum : +.Bl -tag -width indent +.It Fl b , Fl -binary +Read files in binary mode. +.It Fl c , Fl -check +The file passed as arguments must contain digest lines generated by the same +digest algorithm in either classical BSD format or in GNU coreutils format. +A line with the file name followed by a colon +.Dq ":" +and either OK or FAILED is written for each well-formed line in the digest file. +If applicable, the number of failed comparisons and the number of lines that were +skipped since they were not well-formed are printed at the end. +The +.Fl -quiet +option can be used to quiesce the output unless there are mismatched entries in +the digest. +.It Fl -help +Print a usage message and exit. +.It Fl -ignore-missing +When verifying checksums, ignore files for which checksums are given +but which aren't found on disk. +.It Fl -quiet +When verifying checksums, do not print anything unless the +verification fails. +.It Fl -status +When verifying checksums, do not print anything at all. +The exit code will reflect whether verification succeeded. +.It Fl -strict +When verifying checksums, fail if the input is malformed. +.It Fl -tag +Produce BSD-style output. +.It Fl t , Fl -text +Read files in text mode. +This is the default. +Note that this implementation does not differentiate between binary +and text mode. +.It Fl -version +Print version information and exit. +.It Fl w , Fl -warn +When verifying checksums, warn about malformed input. +.It Fl z , Fl -zero +Terminate output lines with NUL rather than with newline. +.El +.Ss PERL OPTIONS +The following options are available in Perl mode, i.e. when the program +is invoked with the name +.Dq shasum : +.Bl -tag -width indent +.It Fl 0 , Fl -01 +Read files in bits mode: ASCII +.Sq 0 +and +.Sq 1 +characters correspond to 0 and 1 bits, respectively, and all other +characters are ignored. +See +.Sx BUGS . +.It Fl a Ar alg , Fl -algorithm Ar alg +Use the specified algorithm: +.Dq 1 +for SHA-1 (default), +.Dq xxx +for +.Va xxx Ns -bit +SHA-2 (e.g. +.Dq 256 +for SHA-256) +or +.Dq xxxyyy +for +.Va xxx Ns -bit +SHA-2 truncated to +.Va yyy +bits (e.g. +.Dq 512224 +for SHA-512/224). +.It Fl b , Fl -binary +Read files in binary mode. +.It Fl c , Fl -check +The file passed as arguments must contain digest lines generated by the same +digest algorithm in either classical BSD format or in GNU coreutils format. +A line with the file name followed by a colon +.Dq ":" +and either OK or FAILED is written for each well-formed line in the digest file. +If applicable, the number of failed comparisons and the number of lines that were +skipped since they were not well-formed are printed at the end. +The +.Fl -quiet +option can be used to quiesce the output unless there are mismatched entries in +the digest. +.It Fl -help +Print a usage message and exit. +.It Fl -ignore-missing +When verifying checksums, ignore files for which checksums are given +but which aren't found on disk. +.It Fl -quiet +When verifying checksums, do not print anything unless the +verification fails. +.It Fl -status +When verifying checksums, do not print anything at all. +The exit code will reflect whether verification succeeded. +.It Fl -strict +When verifying checksums, fail if the input is malformed. +.It Fl -tag +Produce BSD-style output. +.It Fl t , Fl -text +Read files in text mode. +This is the default. +Note that this implementation does not differentiate between binary +and text mode. +.It Fl U , Fl -UNIVERSAL +Read files in universal mode: any CR-LF pair, as well as any CR not +followed by LF, is translated to LF before the digest is computed. +.It Fl -version +Print version information and exit. +.It Fl w , Fl -warn +When verifying checksums, warn about malformed input. +.El .Sh EXIT STATUS The -.Nm md5 , sha1 , sha224 , sha256 , sha512 , sha512t256 , rmd160 , -.Nm skein256 , skein512 , +.Nm md5 , sha1 , sha224 , sha256 , sha512 , sha512t224 , sha512t256 , +.Nm rmd160 , skein256 , skein512 , and .Nm skein1024 utilities exit 0 on success, @@ -149,6 +282,16 @@ utilities exit 0 on success, and 2 if at least one file does not have the same hash as the .Fl c option. +.Pp +The +.Nm md5sum , sha1sum , sha224sum , sha256sum , sha512sum , +.Nm sha512t224sum , sha512t256sum , +.Nm rmd160 , skein256 , skein512 , skein1024 +and +.Nm shasum +utilities exit 0 on success and 1 if at least one of the input files +could not be read or, when verifying checksums, does not have the +expected checksum. .Sh EXAMPLES Calculate the MD5 checksum of the string .Dq Hello . @@ -169,11 +312,22 @@ Calculate the checksum of multiple files reversing the output: $ md5 -r /boot/loader.conf /etc/rc.conf ada5f60f23af88ff95b8091d6d67bef6 /boot/loader.conf d80bf36c332dc0fdc479366ec3fa44cd /etc/rc.conf -.Pd -The -.Nm -sum -variants put 2 blank characters between hash and file name for full compatibility -with the coreutils versions of these commands. +.Ed +.Pp +This is almost but not quite identical to the output from GNU mode: +.Bd -literal -offset indent +$ md5sum /boot/loader.conf /etc/rc.conf +ada5f60f23af88ff95b8091d6d67bef6 /boot/loader.conf +d80bf36c332dc0fdc479366ec3fa44cd /etc/rc.conf +.Ed +.Pp +Note the two spaces between hash and file name. +If binary mode is requested, they are instead separated by a space and +an asterisk: +.Bd -literal -offset indent +$ md5sum -b /boot/loader.conf /etc/rc.conf +ada5f60f23af88ff95b8091d6d67bef6 */boot/loader.conf +d80bf36c332dc0fdc479366ec3fa44cd */etc/rc.conf .Ed .Pp Write the digest for @@ -197,9 +351,7 @@ $ md5 -c randomstring /boot/loader.conf MD5 (/boot/loader.conf) = ada5f60f23af88ff95b8091d6d67bef6 [ Failed ] .Ed .Pp -If invoked with a name ending in -.Nm -sum -the +In GNU mode, the .Fl c option does not compare against a hash string passed as parameter. Instead, it expects a digest file, as created under the name @@ -212,11 +364,12 @@ $ md5 -c digest /boot/loader.conf /boot/loader.conf: OK .Ed .Pp -The digest file may contain any number of lines in the format generated with or without the -.Fl r -option -.Pq i.e., in either classical BSD format or in GNU coreutils format . -If a hash value does not match the file, FAILED is printed instead of OK. +The digest file may contain any number of lines in the format +generated in either BSD or GNU mode. +If a hash value does not match the file, +.Dq FAILED +is printed instead of +.Dq OK . .Sh SEE ALSO .Xr cksum 1 , .Xr md5 3 , @@ -252,13 +405,29 @@ Secure Hash Standard (SHS): The RIPEMD-160 page: .Pa https://homes.esat.kuleuven.be/~bosselae/ripemd160.html .Sh BUGS -All of the utilities that end in -.Sq sum -are intended to be compatible with the GNU coreutils programs. -However, the long option functionality is not provided. +In bits mode, the original +.Nm shasum +script is capable of processing inputs of arbitrary length. +This implementation is not, and will issue an error if the input +length is not a multiple of eight bits. .Sh ACKNOWLEDGMENTS -This program is placed in the public domain for free general use by -RSA Data Security. +.An -nosplit +This utility was originally derived from a program which was placed in +the public domain for free general use by RSA Data Security. .Pp -Support for SHA-1 and RIPEMD-160 has been added by +Support for SHA-1 and RIPEMD-160 was added by .An Oliver Eikemeier Aq Mt eik@FreeBSD.org . +.Pp +Support for SHA-2 was added by +.An Colin Percival Aq Mt cperciva@FreeBSD.org +and +.An Allan Jude Aq Mt allanjude@FreeBSD.org . +.Pp +Support for SKEIN was added by +.An Allan Jude Aq Mt allanjude@FreeBSD.org . +.Pp +Compatibility with GNU coreutils was added by +.An Warner Losh Aq Mt imp@FreeBSD.org +and much expanded by +.An Dag-Erling Sm\(/orgrav Aq Mt des@FreeBSD.org , +who also added Perl compatibility. diff --git a/sbin/md5/md5.c b/sbin/md5/md5.c index 6bc1a780df86..98cfb37110d5 100644 --- a/sbin/md5/md5.c +++ b/sbin/md5/md5.c @@ -21,11 +21,13 @@ __FBSDID("$FreeBSD$"); #include -#include #include +#include +#include #include #include +#include #include #include #include @@ -54,16 +56,20 @@ __FBSDID("$FreeBSD$"); #define TEST_BLOCK_COUNT 100000 #define MDTESTCOUNT 8 -static int bflag; -static int cflag; -static int pflag; -static int qflag; -static int rflag; -static int sflag; -static int skip; +static char *progname; + +static bool cflag; +static bool pflag; +static bool qflag; +static bool sflag; +static bool wflag; +static bool strict; +static bool skip; +static bool ignoreMissing; static char* checkAgainst; static int checksFailed; -static int failed; +static bool failed; +static int endl = '\n'; typedef void (DIGEST_Init)(void *); typedef void (DIGEST_Update)(void *, const unsigned char *, size_t); @@ -84,21 +90,22 @@ extern const char *SKEIN1024_TestOutput[MDTESTCOUNT]; typedef struct Algorithm_t { const char *progname; + const char *perlname; const char *name; const char *(*TestOutput)[MDTESTCOUNT]; DIGEST_Init *Init; DIGEST_Update *Update; DIGEST_End *End; char *(*Data)(const void *, unsigned int, char *); - char *(*Fd)(int, char *); } Algorithm_t; static void MD5_Update(MD5_CTX *, const unsigned char *, size_t); -static void MDOutput(const Algorithm_t *, char *, char **); +static char *MDInput(const Algorithm_t *, FILE *, char *, bool); +static void MDOutput(const Algorithm_t *, char *, const char *); static void MDTimeTrial(const Algorithm_t *); static void MDTestSuite(const Algorithm_t *); -static char *MDFilter(const Algorithm_t *, char*, int); static void usage(const Algorithm_t *); +static void version(void); typedef union { MD5_CTX md5; @@ -121,47 +128,155 @@ typedef union { /* algorithm function table */ static const struct Algorithm_t Algorithm[] = { - { "md5", "MD5", &MD5TestOutput, (DIGEST_Init*)&MD5Init, + { "md5", NULL, "MD5", + &MD5TestOutput, (DIGEST_Init*)&MD5Init, (DIGEST_Update*)&MD5_Update, (DIGEST_End*)&MD5End, - &MD5Data, &MD5Fd }, - { "sha1", "SHA1", &SHA1_TestOutput, (DIGEST_Init*)&SHA1_Init, + &MD5Data }, + { "sha1", "1", "SHA1", + &SHA1_TestOutput, (DIGEST_Init*)&SHA1_Init, (DIGEST_Update*)&SHA1_Update, (DIGEST_End*)&SHA1_End, - &SHA1_Data, &SHA1_Fd }, - { "sha224", "SHA224", &SHA224_TestOutput, (DIGEST_Init*)&SHA224_Init, + &SHA1_Data }, + { "sha224", "224", "SHA224", + &SHA224_TestOutput, (DIGEST_Init*)&SHA224_Init, (DIGEST_Update*)&SHA224_Update, (DIGEST_End*)&SHA224_End, - &SHA224_Data, &SHA224_Fd }, - { "sha256", "SHA256", &SHA256_TestOutput, (DIGEST_Init*)&SHA256_Init, + &SHA224_Data }, + { "sha256", "256", "SHA256", + &SHA256_TestOutput, (DIGEST_Init*)&SHA256_Init, (DIGEST_Update*)&SHA256_Update, (DIGEST_End*)&SHA256_End, - &SHA256_Data, &SHA256_Fd }, - { "sha384", "SHA384", &SHA384_TestOutput, (DIGEST_Init*)&SHA384_Init, + &SHA256_Data }, + { "sha384", "384", "SHA384", + &SHA384_TestOutput, (DIGEST_Init*)&SHA384_Init, (DIGEST_Update*)&SHA384_Update, (DIGEST_End*)&SHA384_End, - &SHA384_Data, &SHA384_Fd }, - { "sha512", "SHA512", &SHA512_TestOutput, (DIGEST_Init*)&SHA512_Init, + &SHA384_Data }, + { "sha512", "512", "SHA512", + &SHA512_TestOutput, (DIGEST_Init*)&SHA512_Init, (DIGEST_Update*)&SHA512_Update, (DIGEST_End*)&SHA512_End, - &SHA512_Data, &SHA512_Fd }, - { "sha512t224", "SHA512t224", &SHA512t224_TestOutput, (DIGEST_Init*)&SHA512_224_Init, + &SHA512_Data }, + { "sha512t224", "512224", "SHA512t224", + &SHA512t224_TestOutput, (DIGEST_Init*)&SHA512_224_Init, (DIGEST_Update*)&SHA512_224_Update, (DIGEST_End*)&SHA512_224_End, - &SHA512_224_Data, &SHA512_224_Fd }, - { "sha512t256", "SHA512t256", &SHA512t256_TestOutput, (DIGEST_Init*)&SHA512_256_Init, + &SHA512_224_Data }, + { "sha512t256", "512256", "SHA512t256", + &SHA512t256_TestOutput, (DIGEST_Init*)&SHA512_256_Init, (DIGEST_Update*)&SHA512_256_Update, (DIGEST_End*)&SHA512_256_End, - &SHA512_256_Data, &SHA512_256_Fd }, - { "rmd160", "RMD160", &RIPEMD160_TestOutput, + &SHA512_256_Data }, + { "rmd160", NULL, "RMD160", + &RIPEMD160_TestOutput, (DIGEST_Init*)&RIPEMD160_Init, (DIGEST_Update*)&RIPEMD160_Update, - (DIGEST_End*)&RIPEMD160_End, &RIPEMD160_Data, &RIPEMD160_Fd }, - { "skein256", "Skein256", &SKEIN256_TestOutput, + (DIGEST_End*)&RIPEMD160_End, &RIPEMD160_Data }, + { "skein256", NULL, "Skein256", + &SKEIN256_TestOutput, (DIGEST_Init*)&SKEIN256_Init, (DIGEST_Update*)&SKEIN256_Update, - (DIGEST_End*)&SKEIN256_End, &SKEIN256_Data, &SKEIN256_Fd }, - { "skein512", "Skein512", &SKEIN512_TestOutput, + (DIGEST_End*)&SKEIN256_End, &SKEIN256_Data }, + { "skein512", NULL, "Skein512", + &SKEIN512_TestOutput, (DIGEST_Init*)&SKEIN512_Init, (DIGEST_Update*)&SKEIN512_Update, - (DIGEST_End*)&SKEIN512_End, &SKEIN512_Data, &SKEIN512_Fd }, - { "skein1024", "Skein1024", &SKEIN1024_TestOutput, + (DIGEST_End*)&SKEIN512_End, &SKEIN512_Data }, + { "skein1024", NULL, "Skein1024", + &SKEIN1024_TestOutput, (DIGEST_Init*)&SKEIN1024_Init, (DIGEST_Update*)&SKEIN1024_Update, - (DIGEST_End*)&SKEIN1024_End, &SKEIN1024_Data, &SKEIN1024_Fd } + (DIGEST_End*)&SKEIN1024_End, &SKEIN1024_Data }, + { } }; -static unsigned digest; -static unsigned malformed; -static bool gnu_emu = false; +static int digest = -1; +static unsigned int malformed; + +static enum mode { + mode_bsd, + mode_gnu, + mode_perl, +} mode = mode_bsd; + +static enum input_mode { + input_binary = '*', + input_text = ' ', + input_universal = 'U', + input_bits = '^', +} input_mode = input_binary; + +static enum output_mode { + output_bare, + output_tagged, + output_reverse, + output_gnu, +} output_mode = output_tagged; + +enum optval { + opt_end = -1, + /* ensure we don't collide with shortopts */ + opt_dummy = CHAR_MAX, + /* BSD options */ + opt_check, + opt_passthrough, + opt_quiet, + opt_reverse, + opt_string, + opt_time_trial, + opt_self_test, + /* GNU options */ + opt_binary, + opt_help, + opt_ignore_missing, + opt_status, + opt_strict, + opt_tag, + opt_text, + opt_warn, + opt_version, + opt_zero, + /* Perl options */ + opt_algorithm, + opt_bits, + opt_universal, +}; + +static const struct option bsd_longopts[] = { + { "check", required_argument, 0, opt_check }, + { "passthrough", no_argument, 0, opt_passthrough }, + { "quiet", no_argument, 0, opt_quiet }, + { "reverse", no_argument, 0, opt_reverse }, + { "string", required_argument, 0, opt_string }, + { "time-trial", no_argument, 0, opt_time_trial }, + { "self-test", no_argument, 0, opt_self_test }, + { } +}; +static const char *bsd_shortopts = "bc:pqrs:tx"; + +static const struct option gnu_longopts[] = { + { "binary", no_argument, 0, opt_binary }, + { "check", no_argument, 0, opt_check }, + { "help", no_argument, 0, opt_help }, + { "ignore-missing", no_argument, 0, opt_ignore_missing }, + { "quiet", no_argument, 0, opt_quiet }, + { "status", no_argument, 0, opt_status }, + { "strict", no_argument, 0, opt_strict }, + { "tag", no_argument, 0, opt_tag }, + { "text", no_argument, 0, opt_text }, + { "version", no_argument, 0, opt_version }, + { "warn", no_argument, 0, opt_warn }, + { "zero", no_argument, 0, opt_zero }, + { } +}; +static const char *gnu_shortopts = "bctwz"; + +static const struct option perl_longopts[] = { + { "algorithm", required_argument, 0, opt_algorithm }, + { "check", required_argument, 0, opt_check }, + { "help", no_argument, 0, opt_help }, + { "ignore-missing", no_argument, 0, opt_ignore_missing }, + { "quiet", no_argument, 0, opt_quiet }, + { "status", no_argument, 0, opt_status }, + { "strict", no_argument, 0, opt_strict }, + { "tag", no_argument, 0, opt_tag }, + { "text", no_argument, 0, opt_text }, + { "UNIVERSAL", no_argument, 0, opt_universal }, + { "version", no_argument, 0, opt_version }, + { "warn", no_argument, 0, opt_warn }, + { "01", no_argument, 0, opt_bits }, + { } +}; +static const char *perl_shortopts = "0a:bchqstUvw"; static void MD5_Update(MD5_CTX *c, const unsigned char *data, size_t len) @@ -177,60 +292,70 @@ struct chksumrec { static struct chksumrec *head = NULL; static struct chksumrec **next = &head; +static unsigned int numrecs; #define PADDING 7 /* extra padding for "SHA512t256 (...) = ...\n" style */ #define CHKFILELINELEN (HEX_DIGEST_LENGTH + MAXPATHLEN + PADDING) -static int gnu_check(const char *checksumsfile) +static void +gnu_check(const char *checksumsfile) { FILE *inp; - char linebuf[CHKFILELINELEN]; - int linelen; + char *linebuf = NULL; + size_t linecap; + ssize_t linelen; int lineno; char *filename; char *hashstr; struct chksumrec *rec; const char *digestname; - int digestnamelen; - int hashstrlen; + size_t digestnamelen; + size_t hashstrlen; - if ((inp = fopen(checksumsfile, "r")) == NULL) + if (strcmp(checksumsfile, "-") == 0) + inp = stdin; + else if ((inp = fopen(checksumsfile, "r")) == NULL) err(1, "%s", checksumsfile); digestname = Algorithm[digest].name; digestnamelen = strlen(digestname); hashstrlen = strlen(*(Algorithm[digest].TestOutput[0])); - lineno = 1; - while (fgets(linebuf, sizeof(linebuf), inp) != NULL) { - linelen = strlen(linebuf) - 1; - if (linelen <= 0) - break; - if (linebuf[linelen] != '\n') - errx(1, "malformed input line %d (len=%d)", lineno, linelen); + lineno = 0; + linecap = CHKFILELINELEN; + while ((linelen = getline(&linebuf, &linecap, inp)) > 0) { + lineno++; + while (linelen > 0 && linebuf[linelen - 1] == '\n') + linelen--; linebuf[linelen] = '\0'; filename = linebuf + digestnamelen + 2; hashstr = linebuf + linelen - hashstrlen; /* * supported formats: * BSD: (): - * GNU: [ *] + * GNU: [ *U^] */ - if (linelen >= digestnamelen + hashstrlen + 6 && + if ((size_t)linelen >= digestnamelen + hashstrlen + 6 && strncmp(linebuf, digestname, digestnamelen) == 0 && strncmp(filename - 2, " (", 2) == 0 && - strncmp(hashstr - 4, ") = ", 4) == 0) { + strncmp(hashstr - 4, ") = ", 4) == 0 && + strspn(hashstr, "0123456789ABCDEFabcdef") == hashstrlen) { *(hashstr - 4) = '\0'; - } else if (linelen >= hashstrlen + 3 && + } else if ((size_t)linelen >= hashstrlen + 3 && + strspn(linebuf, "0123456789ABCDEFabcdef") == hashstrlen && linebuf[hashstrlen] == ' ') { linebuf[hashstrlen] = '\0'; hashstr = linebuf; filename = linebuf + hashstrlen + 1; - if (*filename == ' ' || *filename == '*') - filename++; } else { + if (wflag) { + warnx("%s: %d: improperly formatted " + "%s checksum line", + checksumsfile, lineno, + mode == mode_perl ? "SHA" : digestname); + } malformed++; continue; } - rec = malloc(sizeof (*rec)); + rec = malloc(sizeof(*rec)); if (rec == NULL) errx(1, "malloc failed"); rec->chksum = strdup(hashstr); @@ -240,10 +365,10 @@ static int gnu_check(const char *checksumsfile) rec->next = NULL; *next = rec; next = &rec->next; - lineno++; + numrecs++; } - fclose(inp); - return (lineno - 1); + if (inp != stdin) + fclose(inp); } /* Main driver. @@ -261,18 +386,19 @@ main(int argc, char *argv[]) #ifdef HAVE_CAPSICUM cap_rights_t rights; #endif - int ch, fd; - char *p, *string; + const struct option *longopts; *** 1830 LINES SKIPPED ***