Slow resilvering with mirrored ZIL

[Gezeala M. Bacuño II]

On Wed, Jul 17, 2013 at 10:35 AM, Warren Block <wblock at wonkity.com> wrote:

> On Wed, 17 Jul 2013, Gezeala M. Bacuño II wrote:
>
>  If ZFS goes on a bare drive, it will be aligned by default.  If ZFS is
>> going in a partition, yes, align that partition to 4K boundaries or larger
>> multiples of 4K, like 1M.
>>
>> Your statement is enlightening and concise, exactly what I need. Thanks.
>>
>>       The gnop/ashift workaround is just to get ZFS to use the right
>> block size.  So if you don't take care to get partition alignment right,
>> you might end up using the right
>>       block size but misaligned.
>>
>>       And yes, it will be nice to be able to just explicitly tell ZFS the
>> block size to use.
>>
>>
>> We do add the entire drive (no partitions) to ZFS, perform gnop/ashift
>> and other necessary steps and then verify ashift=12 through zdb.
>>
>> The gpart/gnop/ashift steps, if I understand correctly (do correct me if
>> I'm stating this incorrectly), is needed for further SSD performance
>> tuning. Taking into consideration leaving a
>> certain chunk for wear leveling and also if the SSD has a size that may
>> be too big for L2ARC.
>>
>
> Well, there are several things going on.
>
> Partitions can be used for a couple of things.  Limiting the size of space
> available to ZFS, leaving an unallocated part of the drive for wear
> leveling.  Note that ZFS on FreeBSD now has TRIM, which should make leaving
> unused space on SSDs unnecessary.
>
> Aligning partitions preserves performance.  If a partition is misaligned,
> writes can slow down to half speed.  For example, a 4K filesystem block
> written to an aligned partition writes a single block. If the partition is
> misaligned, that 4K write is split over two disk blocks.  Each block has to
> be read, partly modified, then written, taking roughly twice as long.
>
> Finally, ZFS's ashift controls the minimum size of block ZFS uses.
> ashift=12 (12 bits) sets that to 4K blocks (2^12=4096).  Again, a
> performance thing, matching the filesystem block size to device block size.
>
> It would be interesting to see a benchmark of ZFS on a 4K drive with
> different ashift values.


Right on again. I forgot to include on my reply, that it is for a specific
use case similar to ours, wherein we dedicate the entire drive to the pool.
I believe it is totally time to put all these howto/faq stuff on a central
FreeBSD repository, I think there's another thread requesting for the same
thing.

Scenarios:
a] maximizing pool and drive size, don't need to partition -- these are the
steps.
     a.1] new drives
     a.2] used drives
b] For those with limited drives, limited enclosures etc -- these are the
steps you may want to check out
c] zfs-on-root
d] and so on..

This will help a lot on deciding which steps to follow and which are
necessary or not, therefore, avoiding all these repeated questions (just
like mine) on ZFS setup/performance/tuning.

https://wiki.freebsd.org/ZFSTuningGuide (WIP) -  outdated, and there's no
section for initial zpool/drive(s) setup.