3ware RAID ctrlr: twe0: twe_map_request: malloc failed.

[Scott Long]

Howard Goldstein wrote:
> Scott Long wrote:
>> Howard Goldstein wrote:
>>> Would someone confirm (or disabuse!) me of the belief that these 3ware
>>> Escalade 8106-LP2 (2 port sata raid controllers) related messages are
>>> informational and the UFS device is retrying after it sees ENOMEM
>>> causing these?
>>>
>>> Aug 13 10:29:48 cally kernel: twe0: twe_map_request: malloc failed
>>> Aug 13 10:33:04 cally kernel: twe0: twe_map_request: malloc failed
>>> Aug 13 11:05:37 cally kernel: twe0: twe_map_request: malloc failed
>>> Aug 13 11:14:55 cally kernel: twe0: twe_map_request: malloc failed
>>> Aug 13 12:07:44 cally kernel: twe0: twe_map_request: malloc failed
>>> Aug 13 12:28:05 cally kernel: twe0: twe_map_request: malloc failed
>>> Aug 13 14:29:59 cally kernel: twe0: twe_map_request: malloc failed
>>>
>>> (Either way but particularly in the event the data are being lost is
>>> there a sysctl or kernel build option to supply a bit more memory in
>>> advance?)
>> The system is doing unaligned I/O to the array (either via an app that
>> has a file opened O_DIRECT, or via I/O directly to the device node), and
>> there is corresponding memory pressure that is making the driver fail
>> at the re-alignment process.  The problem is likely temporary and the
>> operation eventually succeeds after a few retries, but success is not
>> guaranteed, and failure will almost certainly cause system instability.
>> The fix is to rewrite the alignment code in the driver to use the
>> automatic, failure-free, alignment service that busdma provides.  I'm
>> happy to provide direction to anyone who wants to tackle this.
> 
> Alright, I like this idea since rebuilding the devices to avoid what I
> thought were unaligned access didn't work and I'm still seeing these
> messages and a performance hit I don't understand.  I imagine I'll get
> better performance too if the driver can avoid doing its own alignment
> which probably isn't as optmized (?)  The busdma man page looks nice and
> complete too - cool
> 
> Could you point me to a model driver or two already using the busdma stuff?
> 

Sorry for the confusion, I'm talking about aligned memory accesses, not 
disk accesses.  The TWE architecture requires that all transfers be on
4KB boundaries in memory.

I'll write up some details on the work tonight.  Basically it involves 
removing the manual aligment code and modifying the bus_dma tags to 
specify 4k alignment.  It sounds easy, but I seem to recall something
nasty in the code the last time I looked at it.

Scott