processes not getting fair share of available disk I/O (was: Re: TCP parameters and interpreting tcpdump output )

[Dieter]

In message <20061123011751.GA45406 at xor.obsecurity.org>, Kris Kennaway writes:
> 
> --mP3DRpeJDSE+ciuQ
> Content-Type: text/plain; charset=us-ascii
> Content-Disposition: inline
> Content-Transfer-Encoding: quoted-printable
> 
> On Wed, Nov 22, 2006 at 11:02:54AM +0000, Dieter wrote:
> > In message <20061122165238.GA37819 at xor.obsecurity.org>, Kris Kennaway wri=
> tes:
> >=20
> > > > > I'm surprised that you're seeing that much of a "hang".  Even if th=
> e di=3D
> > > sks
> > > > > are busy, the system should slow down all disk processes equally, s=
> o no
> > > > > one process "blocks", but they're all a little slower.
> > > >=3D20
> > > > I collected a bit of data:
> > > >=3D20
> > > > While copying a large file from disk1 to disk2,
> > > >=3D20
> > > > time ls on a small directory on disk3 (not cached in memory)
> > > >=3D20
> > > > real    0m0.032s
> > > > user    0m0.000s
> > > > sys     0m0.003s
> > > >=3D20
> > > > time ls on a small directory on disk2
> > > >=3D20
> > > > real    4m51.911s
> > > > user    0m0.000s
> > > > sys     0m0.002s
> > > >=3D20
> > > > I expect access to a busy disk to take longer, but 5 minutes is
> > > > a bit much.  And that's the root directory of the filesystem,
> > > > it didn't have to follow a long chain of directories to get there.
> > > >=3D20
> > > > Sometimes I see long delays when accessing disk3, but it is
> > > > behaving at the moment.
> > >=20
> > > ls still has to acquire a number of locks in order to be sure that the
> > > contents of the directory aren't changing.  If there are lots of other
> > > processes all competing for these locks, it will be slow.  It looks
> > > like that's the case on your system, although details of your workload
> > > have been trimmed from your email.
> >=20
> > In telnet window 1:
> >=20
> > cd /disk1/
> > cp -ip very_big_file /disk2/bar/	(the workload)
> >=20
> > In telnet window 2:
> >=20
> > time ls /disk3/foo1/  (make sure time and ls are cached in memory)
> > time ls /disk3/foo2/  (see timing numbers above)
> > time ls /disk2/       (see timing numbers above)
> >=20
> > The /disk2/ directory is small, only contains 3 directories and .snap
> >=20
> > Would the cp into /disk2/bar/ lock the /disk2/ directory?
> 
> It shouldn't do.
> 
> What scheduler are you using?

kern.sched.name: 4BSD
kern.sched.quantum: 100000
kern.sched.preemption: 1