11-STABLE vs 11.0-RELENG test

[Alexander Motin]

On 06.12.2016 22:26, Alexander Motin wrote:
> I've reproduced this issue with quick test on my lab system configured
> with 12-disk RAIDZ2 pool.  I've measured write and read back (with and
> without prefetch) speeds for pool recreated on different FreeBSD head
> revisions:
> 		r309625	r305456	r305330	r305322
> write		702	701	1115	1120
> read w/ pref	232	228	518	512
> read w/o pref	128	126	242	240
> 
> I suspect we could obtain the problem here:
> 
> r305331 | mav | 2016-09-03 13:04:37 +0300 (сб, 03 сент. 2016) | 45 lines
> 
> MFV r304155: 7090 zfs should improve allocation order and throttle
> allocations

Closer look shown me the cause.  This code sorts I/Os on time, offset
and memory address.  But time on FreeBSD (to reduce overhead) returned
with 1ms resolution, so it does not provide reliable ordering.  Offset
sorting used by this patch is broken by design, since io_offset field is
always zero there, since it is used only for physical I/Os, not for
logical.  As result, I/Os are "sorted" on memory address, that in fact
means complete randomization of all allocations within one millisecond,
predictably killing read performance.

Switching gethrtime() emulation from getnanouptime() to nanouptime()
fixes the read performance, resulting:
		nanouptime()
write		702
read w/ pref	845
read w/o pref	272

It would be good to make offset sorting really work there rather then
just switching to high resolution time source, but that maybe quite
invasive.  Will look more.

-- 
Alexander Motin