RELENG_7 heavy disk = system crawls

[Dieter]

> Hi. I'm running RELENG_7 on an older single P4. It has a lot of
> disk on it that does mainly sequential read/write of gigs of data.

> In short, whenever I'm doing sequential disk stuff, human interface
> system performance tanks, big time.

> I used to do similar stuff on RELENG_4 on an old dual PII with plain
> UFS and I could run gigs of disk data, all the spindles, 8 of them,
> through sha1, all at once, and still have good interactive performance.
> 
> Yet fire off one sequential read/write, perhaps piped into sha1 for
> more cpu load and it's game over. I can renice the user process way
> down into idprio with no effect. So I doubt giving rtprio to all the
> human processes would do anything.
> 
> Is RELENG_7 or GELI+ZFS the dog here? Top of course shows
> GELI+ZFS eating all the cpu. Ok, fine, so how do I say, hey, I don't
> care about disk, it'll finish eventually, just give me my gui and
> keystrokes back?
> 
> It just doesn't seem right that this one subsystem should be
> monopolizing the cpu this way? Help/cluebat? Thanks.

Are you *sure* that the cpu is your bottleneck?  I see problems
where sequential read/write of gigs of data on one disk interferes
with i/o to other disks, and it isn't a cpu problem in my case.
Just the opposite, cpu bound jobs don't create the problem, i/o
bound jobs do.  As expected, idprio doesn't help my problem either.
I suspect that the problem is the i/o bound job is hogging the disk
cache or some similar resource.  I run FFS, no ZFS, and I've seen
the same problem on 6.0, 6.2, 7.0 and 7.1.

Unix grew up on machines where cpu was always the scarce resource,
so nice only cares about cpu.  Today i/o is often the scarce
resource, so we need a way to nice i/o up/down similar to nicing
cpu up/down.

FreeBSD is supposed to be the "performance" BSD, but watching
it leave several disks idle while one disk does i/o makes me
wonder.