ZFS confusion

Mon Jan 27 12:12:07 UTC 2014

On Mon, 27 Jan 2014 11:42-0000, Kaya Saman wrote:

> Many thanks everyone (Trond, Dennis, Steve)!!
> 
> So RAIDz2 or 3 is going to be preferred per the advice given.
> 
> Now I just need to figure out how to make that work best with my current 5
> block of disks....  perhaps wait for a while then add some more disks in the
> mix then create the raidz(x) platform??
> 
> It would be really good if raidz could be expandable, ie by adding extra 'new'
> disks into the same vdev.

It's there!

Try: zpool attach <pool_name> <existing_member> <new_member1> [new_member2 ...]

> Anyhow, luckily I have multiple backups of everything in any case; just flying
> around on loose disks though :-(
> 
> 
> Regards,
> 
> Kaya
> 
> On 01/27/2014 11:20 AM, Trond Endrestøl wrote:
> > On Mon, 27 Jan 2014 09:59-0000, Kaya Saman wrote:
> > 
> > > Many thanks Trond for the response....
> > > 
> > > I was getting worried for a sec there :-)
> > > 
> > > On 01/27/2014 09:08 AM, Trond Endrestøl wrote:
> > > > On Sat, 25 Jan 2014 19:12-0000, Kaya Saman wrote:
> > > > 
> > > > [...]
> > > > No one's answered this, so I'll just give you my 2 cents.
> > > > 
> > > > Triple parity means you're using storage capacity equivalent of three
> > > > drives for parity alone. If you use five drives in total, this gives
> > > > you 2 drives worth of real data and 3 drives worth of parity. In other
> > > > words, you should really consider using a lot more drives when using
> > > > triple parity, say nine drives.
> > > So, basically if I have (just as silly example); 5x drives of 100MB
> > > capacity,
> > > this would equal 500MB in total.
> > > 
> > > Meaning that only 200MB would be usable.....
> > Also remember a couple of things:
> > 
> > ZFS uses dynamically stripe widths, not fixed ones as HW RAID. A 37K
> > file will occupy 37K, rounded up to the nearest integer of disk
> > blocks, plus the space needed for parity and metadata.
> > 
> > Without a dedicated ZIL, ZFS will also use some of the available space
> > for housekeeping, e.g. ZIL.
> > 
> > You shouldn't fill a pool in production completely with data, leaving
> > no room for any COW operations.
> > 
> > > Having just done a quick experiment:
> > > 
> > > zpool create test_pool raidz3 /tmp/disk1 /tmp/disk2 /tmp/disk3 /tmp/disk4
> > > /tmp/disk5
> > > 
> > > I get:
> > > 
> > > test_pool    476M   304K   476M     0%  1.00x  ONLINE  -
> > > 
> > > then writing a 300M file to pool:
> > > 
> > > dd if=/dev/zero of=/test_pool/file bs=300M count=1
> > > dd: /test_pool/file: No space left on device
> > > 
> > > test_pool    476M   396M  79.8M    83%  1.00x  ONLINE  -
> > > 
> > > du -sch /test_pool/file
> > > 157M    /test_pool/file
> > > 157M    total
> > > 
> > > So I guess, now I have to ask; when building a disk pool, what's the best
> > > option to go for between capacity and redundancy?
> > If you value your data and your data grows, as it usually does, go for
> > both. I hope you're adding some form of backup to the mix, as ZFS is
> > no substitute for doing proper backups to independent media.
> > 
> > > In my case I gradually want to expand my storage when I need it, so to
> > > maximize capacity RAID0 would be the easiest. This doesn't give me any
> > > redundancy however, unless I use RAID1+0 but then I loose physical
> > > capacity
> > > due to the mirrors.
> > > 
> > > 
> > > Would it be better to create a raidz1 pool then just add raidz1 pools to
> > > the
> > > master as time goes by?
> > > 
> > > As in:
> > > 
> > > zpool create test_pool raidz1 /tmp/disk1 /tmp/disk2 /tmp/disk3 /tmp/disk4
> > > /tmp/disk5
> > > 
> > > 
> > > zpool add test_pool raidz1 /tmp/disk6 /tmp/disk7 /tmp/disk8 /tmp/disk9
> > > /tmp/disk10
> > > 
> > > 
> > > dd if=/dev/zero of=/test_pool/file bs=600M count=1                    1+0
> > > records in
> > > 1+0 records out
> > > 629145600 bytes transferred in 190.069986 secs (3310073 bytes/sec)
> > > 
> > > 
> > > Then running a 'zpool list' gets:
> > > 
> > > test_pool    952M   751M   201M    78%  1.00x  ONLINE  -
> > > 
> > > 
> > > So virtually out of:
> > > 
> > > 
> > >      NAME             STATE     READ WRITE CKSUM
> > >      test_pool        ONLINE       0     0     0
> > >        raidz1-0       ONLINE       0     0     0
> > >          /tmp/disk1   ONLINE       0     0     0
> > >          /tmp/disk2   ONLINE       0     0     0
> > >          /tmp/disk3   ONLINE       0     0     0
> > >          /tmp/disk4   ONLINE       0     0     0
> > >          /tmp/disk5   ONLINE       0     0     0
> > >        raidz1-1       ONLINE       0     0     0
> > >          /tmp/disk6   ONLINE       0     0     0
> > >          /tmp/disk7   ONLINE       0     0     0
> > >          /tmp/disk8   ONLINE       0     0     0
> > >          /tmp/disk9   ONLINE       0     0     0
> > >          /tmp/disk10  ONLINE       0     0     0
> > > 
> > > errors: No known data errors
> > > 
> > > 
> > > I have 8 disks usable with 2x disks for parity.... if my understanding is
> > > correct.
> > Correct. One "parity drive" evenly (or close to evenly) spread over
> > the members in each raidz1 vdev.
> > 
> > I just realized having multiple zraidN vdevs reduces the time needed
> > for resilvering. Just don't make each vdev too small, i.e. with too
> > few drives.
> > 
> > As other's have told you, consider raidz2 or raidz3, as drives from
> > the same batch and with equal wear and tear, might fail more or less
> > simultaneously.

-- 
+-------------------------------+------------------------------------+
| Vennlig hilsen,               | Best regards,                      |
| Trond Endrestøl,              | Trond Endrestøl,                   |
| IT-ansvarlig,                 | System administrator,              |
| Fagskolen Innlandet,          | Gjøvik Technical College, Norway,  |
| tlf. mob.   952 62 567,       | Cellular...: +47 952 62 567,       |
| sentralbord 61 14 54 00.      | Switchboard: +47 61 14 54 00.      |
+-------------------------------+------------------------------------+