svn commit: r292955 - head/lib/libmd

[Allan Jude]

On 2015-12-30 23:10, Bruce Evans wrote:
> On Thu, 31 Dec 2015, Bruce Evans wrote:
> 
>>
>> wc /proc/0/* works.  md5 works like wc using a hack to avoid calling this
>> broken function.  E.g.,
>>
>>    for i in $(ls /proc/0/*); do echo -n "$i: "; md5 <$i; done  # gives
>>
>> /proc/0/cmdline: 3c5896b1ac441f4998f052e2126e8d20
>> /proc/0/ctl: d41d8cd98f00b204e9800998ecf8427e
>> /proc/0/etype: 674441960ca1ba2de08ad4e50c9fde98
>> /proc/0/rlimit: 67d6ad67b412e1bceb7cb508a3492197
>> /proc/0/status: 3ccc3067b97c3232ea2dbcb64c458fd4
> 
> Further examples:
> 
>     md5 # on terminal input
> 
> works correctly by not using MDXFileChunk().
> 
>     md5 /dev/stdin # on the same terminal input
> 
> produces the d41d8cd98f00b204e9800998ecf8427e garbage using
> MDXFileChunk().  truss shows that lseek() is broken too -- MDXFileChunk()
> tries it and it succeeds on the unseekable file /dev/stdin.  Bugs in this
> area are common.  E.g., lseek() on named pipes was broken so that it
> succeeded, by rearranging the plumbing use fileops more or less and not
> attaching lseek right.
> 
>     cat | md5 # on the same terminal input
> 
> works correctly by not using MDXFileChunk().
> 
>     cat | md5 /dev/stdin # on the same terminal input
> 
> doesn't work correctly, but it fails better than without the pipe.  Now
> a seek error occurs and is reported as "md5: /dev/stdin: Illegal seek"
> (I can't see where it is reported).  Then md5 exits.  Then cat waits to
> read input.  Then cat fails to write output and is killed by SIGPIPE.
> So md5 handled the seek error in a fail-safe though incorrect way.  One
> correct way is to fall back to the working code, but it is better to
> just use that without an lseek check.
> 
> It is a bug that [l]stat() on /dev/stdin sees the device file and not the
> actual stdin file.  md5 can't work around this except by not using stat().
> 
> Bruce
> 

It seems these problems also slow things down, a lot:

# time md5 /media/md5test/bigdata
MD5 (/media/md5test/bigdata) = 6afad0bf5d8318093e943229be05be67
4.310u 3.476s 0:07.79 99.8%     20+167k 0+0io 0pf+0w
# time env LD_PRELOAD=/usr/obj/media/svn/md5/head/tmp/lib/libmd.so
/usr/obj/media/svn/md5/head/sbin/md5/md5 /media/md5test/bigdata
MD5 (/media/md5test/bigdata) = 6afad0bf5d8318093e943229be05be67
4.133u 0.354s 0:04.49 99.7%     20+167k 1+0io 0pf+0w

(file is fully cached in ZFS ARC, dd reads it at 11GB/s)

Will investigate more tomorrow.

char *
MDXFile(const char *filename, char *buf)
{
	unsigned char buffer[16*1024];
	MDX_CTX ctx;
	int f, i;

	MDXInit(&ctx);
	f = open(filename, O_RDONLY);
	if (f < 0)
		return 0;
	i = 0;
	while (1) {
		i = read(f, buffer, sizeof(buffer));
		if (i <= 0)
			break;
		MDXUpdate(&ctx, buffer, i);
	}

	close(f);
	return (MDXEnd(&ctx, buf));
}


-- 
Allan Jude

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 834 bytes
Desc: OpenPGP digital signature
URL: <http://lists.freebsd.org/pipermail/svn-src-head/attachments/20151231/afc97b8b/attachment.sig>