Trim on gmirrored SSDs is slow and results in inresponsive system

[Steven Hartland]

On 05/03/2015 15:16, Robert Schulze wrote:
> Hi,
>
>> These values look totally reasonable in terms a d/s.
>>
>> How does this compare to the none gmirror case?
> Its barely measurable, but in this case I can start a loop directly
> after unlink with very small interval. The only measurement with
> non-null values is:
>
> dT: 0.205s  w: 0.200s  filter: ada
> L(q) ops/s r/s kBps ms/r w/s kBps ms/w   d/s    kBps  ms/d  %busy Name
>     0     0   0    0  0.0   0    0  0.0     0       0   0.0   0.0  ada0
>     0 51537   5  156  2.4   0    0  0.0 51532 1649018   3.8  18.2  ada1
>
> thats all. It takes not even half a second to push the DELETE ops to the
> SSD in case of non-gmirrored UFS. Again, with a 2 GB file.
That is a high rate, but not inconceivable.
>> Compile a version of the kernel with this change reverted.
>>
>> Details of the change can be seen here.
>> https://svnweb.freebsd.org/base?view=revision&revision=268816
> done, but it did not change behaviour.
>
Thanks, I think we need to do some dtrace instrumentation, to get a 
better picture of what's happening.

If you pick a smaller file, say 10MB what does the following dtrace 
script report for the single disk no gmirror and for a single disk 
gmirror (if thats possible)

#!/usr/sbin/dtrace -s

#pragma D option quiet

enum bio_type {
         BIO_DELETE = 0x04
};

fbt::adastrategy:entry
/((struct bio *)arg0)->bio_cmd == BIO_DELETE/
{
         bp = (struct bio *)arg0;
         periph = (struct cam_periph *)bp->bio_disk->d_drv1;
         softc = (struct ada_softc *)periph->softc;
         printf("%Y ada%d bcount: %d, offset: %d\n", walltimestamp, 
periph->unit_number, bp->bio_bcount, bp->bio_pblkno);
}