processes not getting fair share of available disk I/O

Dieter freebsd at sopwith.solgatos.com
Wed Dec 13 16:39:50 PST 2006


> > Is Giant the only mutex/lock that could be a bottleneck across disks?
> 
> The only one I can think of that is generic.  One would have to do
> more extensive profiling and diagnosis to try and figure out what is
> wrong with your system.

Suggestions of what to look at would be welcome.

> The only explanation that seems to fit is that it's something to do
> with your particular hardware (i.e. driver issue), since it's
> certainly not a problem on general configurations.
> 
> I know that many people have bad things to say about nforce chipsets,
> although I dont know if your particular problem has been reported
> before.

Could APIC have anything to do with this?  It is currently turned off in
firmware.

Today I experimented with vfs.hirunningspace.  If I crank it up, I get
better total write speed with multiple drives doing dd from /dev/zero
to files on disks.  But it doesn't help my real applications, and
in fact appears to hurt them.

I have also discovered that my program that reads from Ethernet and
does non-blocking writes to stdout which get redirected to a disk file
isn't benefiting as much as it should from the large circular buffer.
The buffer should be enough to hold 4.5-5 minutes worth of data, and
is mlock(2)'d into memory.   I can read from four disks at once without
hurting the Ethernet transfer.  But a dd from /dev/zero writing to a
different disk that takes less than 3 seconds interferes with the Ethernet
transfer.  The fcntl(1, O_NONBLOCK); isn't failing, and the write() isn't
returning -1/EAGAIN.


More information about the freebsd-questions mailing list