[zfs] [developer] Re: [smartos-discuss] an interesting survey -- the zpool with most disks you have ever built

[Fred Liu]

2016-03-07 14:04 GMT+08:00 Richard Elling <richard.elling at gmail.com>:

>
> On Mar 6, 2016, at 9:06 PM, Fred Liu <fred.fliu at gmail.com> wrote:
>
>
>
> 2016-03-06 22:49 GMT+08:00 Richard Elling <
> richard.elling at richardelling.com>:
>
>>
>> On Mar 3, 2016, at 8:35 PM, Fred Liu <Fred_Liu at issi.com> wrote:
>>
>> Hi,
>>
>> Today when I was reading Jeff's new nuclear weapon -- DSSD D5's CUBIC
>> RAID introduction,
>> the interesting survey -- the zpool with most disks you have ever built
>> popped in my brain.
>>
>>
>> We test to 2,000 drives. Beyond 2,000 there are some scalability issues
>> that impact failover times.
>> We’ve identified these and know what to fix, but need a real customer at
>> this scale to bump it to
>> the top of the priority queue.
>>
>> [Fred]: Wow! 2000 drives almost need 4~5 whole racks!
>
>>
>> For zfs doesn't support nested vdev, the maximum fault tolerance should
>> be three(from raidz3).
>>
>>
>> Pedantically, it is N, because you can have N-way mirroring.
>>
>
> [Fred]: Yeah. That is just pedantic. N-way mirroring of every disk works
> in theory and rarely happens in reality.
>
>>
>> It is stranded if you want to build a very huge pool.
>>
>>
>> Scaling redundancy by increasing parity improves data loss protection by
>> about 3 orders of
>> magnitude. Adding capacity by striping reduces data loss protection by
>> 1/N. This is why there is
>> not much need to go beyond raidz3. However, if you do want to go there,
>> adding raidz4+ is
>> relatively easy.
>>
>
> [Fred]: I assume you used stripped raidz3 vedvs in your storage mesh of
> 2000 drives. If that is true, the possibility of 4/2000 will be not so low.
>            Plus, reslivering takes longer time if single disk has bigger
> capacity. And further, the cost of over-provisioning spare disks vs raidz4+
> will be an deserved
>             trade-off when the storage mesh at the scale of 2000 drives.
>
>
> Please don't assume, you'll just hurt yourself :-)
> For example, do not assume the only option is striping across raidz3
> vdevs. Clearly, there are many
> different options.
>

[Fred]:  Yeah. Assumptions always go far way from facts! ;-) Is designing a
storage mesh with 2000 drives biz secret? Or it is just too complicate to
elaborate?
Never mind. ;-)

Thanks.

Fred


>
> *smartos-discuss* | Archives
> <https://www.listbox.com/member/archive/184463/=now>
> <https://www.listbox.com/member/archive/rss/184463/22027488-c60da3c5> |
> Modify
> <https://www.listbox.com/member/?member_id=22027488&id_secret=22027488-68892e07>
> Your Subscription <http://www.listbox.com>
>