svn commit: r251372 - in stable/9/sys/cam: ata scsi

[Scott Long]

On Jun 5, 2013, at 9:02 AM, Steven Hartland <killing at multiplay.co.uk> wrote:

> 
> 
> ----- Original Message ----- From: "Justin T. Gibbs" <gibbs at scsiguy.com>
> 
> 
> On Jun 4, 2013, at 3:47, Steven Hartland <smh at FreeBSD.org> wrote:
> 
>> > Author: smh
>> > Date: Tue Jun  4 10:47:44 2013
>> > New Revision: 251372
>> > URL: http://svnweb.freebsd.org/changeset/base/251372
>> > > Log:
>> >  Enhanced BIO_DELETE support for CAM SCSI to add ATA_TRIM support.
>> > >  Disable CAM BIO queue sorting for non-rotating media by default.
>> 
>> Using the elevator makes perfect sense even for random access devices when
>> operations can be merged. Even SSDs have fixed, per-command overheads and
>> merging writes may help prevent fragmentation.  On drivers that support
>> unmapped I/O, merging should be fairly cheap.
>> 
>> It would be an interesting performance experiment.
> 
> In theory I'd agree but in practice this was not the case with bioq
> becoming a noticable bottleneck for SSD based machines, hence this change.
> 
> The sysctl's still allow this to be configured :)
> 

Completely agree.  The bioq_disksort() only serves to burn CPU cycles with the column lock held (thus burning even more CPU cycles elsewhere) once the queue grows to any appreciable size.  Merging of adjacent segments happens (mostly) right now thanks to lock serialization in the issuing path and g_down thread serialization.  Furthermore, bioq_disksort() has no concept of the cost of reads vs writes as they pertain to SSDs.  It's better to have the order be random and/or somewhat separated thanks to operational serialization, than to try extra hard to gets reads and writes close together.

A much better I/O scheduler is needed.  In our tests, bioq_disksort() is worse than no i/o scheduler, so we've turned it off.  We've tried Luigi's scheduler, but it adds too much latency to our workload.  I'm working on a better scheme.

Scott