An order of magnitude higher IOPS needed with ZFS than UFS

[Mark Felder]

On Wed, 12 Jun 2013 06:49:37 -0500, Jeremy Chadwick <jdc at koitsu.org> wrote:

>
> Mark, do you have any references for this?  I'd love to learn/read more
> about this engineering/design aspect (I won't say flaw, I'll just say
> aspect) to ZFS, as it's the first I've heard of it.

Firsthand experience on a couple servers, and some old Sun docs that I  
can't find anymore since Oracle broke the links. If you start googling for  
"ZFS performance 80%" you should come across similar reports. The  
recommendation was always that when you hit about 80% you need to add a  
new vdev or you'll be in serious trouble. I'd always believed that it has  
to do with the way the ZFS COW algorithm works. If my suspicion is correct  
I'd guess it probably stalls trying to find an ideal place to write --  
maybe some cost calculation? I'm reaching for straws now because I don't  
know anything about the code itself.

I'd love to hear from people who have actually touched the code and can  
give a more definitive answer because this does border on "urban legend"  
territory, but I've read it and experienced it a few times so I'm just  
passing it on.