RFC: Suggesting ZFS "best practices" in FreeBSD

[Olivier Smedts]

Hi,

2013/1/22 Kevin Day <toasty at dragondata.com>:
> I run ftpmirror.your.org, which is a 72 x 3TB drive ZFS server. It's a very busy server. It currently houses the only off-site backup of all of the Wikimedia projects(121TB), a full FreeBSD FTP mirror(1T), a full CentOS mirror,  all of FreeBSD-Archive(1.5TB), FreeBSD-CVS, etc. It's usually running between 100 and 1500mbps of ethernet traffic in/out of it. There are usually around 15 FTP connections, 20-50 HTTP connections, 10 rsync connections and 1 or 2 CVS connections.
>
> The only changes we've made that are ZFS specific are atime=off and sync=disabled. Nothing we do uses atimes so disabling that cuts down on a ton of unnecessary writes. Disabling sync is okay here too - we're just mirroring stuff that's available elsewhere, so there's no threat of data loss. Other than some TCP tuning in sysctl.conf, this is running a totally stock kernel with no special settings.

If your workload is mostly made of reads (you're a mirror after all,
you should only write when you're syncing with upstream servers) why
use sync=disabled ? It shouldn't make a big difference for such a
workload.

> I've looked at using an SSD for meta-data only caching, but it appears that we've got far more than 256GB of metadata here that's being accessed regularly (nearly every file is being stat'ed when rsync runs) so I'm guessing it's not going to be incredibly effective unless I buy a seriously large SSD.
>
> If you have any specific questions I'm happy to answer though.
>
> -- Kevin
>
> _______________________________________________
> freebsd-fs at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-fs
> To unsubscribe, send any mail to "freebsd-fs-unsubscribe at freebsd.org"



-- 
Olivier Smedts                                                 _
                                        ASCII ribbon campaign ( )
e-mail: olivier at gid0.org        - against HTML email & vCards  X
www: http://www.gid0.org    - against proprietary attachments / \

  "Il y a seulement 10 sortes de gens dans le monde :
  ceux qui comprennent le binaire,
  et ceux qui ne le comprennent pas."