ZFS makes SSDs faster than memory!

[Attila Nagy]

On 07/23/10 15:29, Ivan Voras wrote:
> On 07/23/10 15:20, Attila Nagy wrote:
>
>    
>> Maybe I should have written this first, but I'm not the only one reading
>> from the machine.
>>      
> You probably realize this makes all your performance data of suspicious
> validity :)
>    
Yes, this is the same I would write to this e-mail, but I can reproduce 
it. :)
Fetching the same file not in the cache three times make the first the 
slowest, the second (after waiting a little to fall out of RAM) the 
fastest and the third the second fastest.
I can consistently reproduce this behaviour, but only via network 
(ftpd/httpd) not from localhost.
>> For random reads even the cheapest MLC outperforms a 7k2 SATA disk (only
>> reads), and this is an Intel stuff, which can do 3000 RIOPS easily.
>>      
>>> Are there any facts backup your assumption that data is really
>>> read from memory, SSD, disk in the named cases?
>>> E.g. by ARC/L2ARC and IO statistics.
>>>
>>>        
>> Yes. When downloading from L2ARC:
>>   L(q)  ops/s    r/s   kBps   ms/r    w/s   kBps   ms/w   %busy Name
>>      0    174    174  21505    0.8      0      0    0.0   13.3| ad4
>>      0    169    169  21479    0.9      0      0    0.0   15.0| ad6
>> when downloading from ARC:
>>   L(q)  ops/s    r/s   kBps   ms/r    w/s   kBps   ms/w   %busy Name
>>      0     26     19   1129    0.6      7     78    0.4    1.3| ad4
>>      0     19     12   1436    1.1      7     78    0.3    1.4| ad6
>>      
> So it looks like you encountered a problem where the memory-based ARC
> cache read performance is incredibly bad?
>    
I wouldn't call it incredibly bad, but it's worse than reading from 
L2ARC (2xSSD), which is pretty strange and not sane, at least to what I 
know about how things work in a computer. :)