zfs home directories best practice

[Rick Macklem]

Graham Allan wrote:
> We're building a new NFS home directory server on FreeBSD with ZFS.
> The
> Solaris ZFS Best Practices docs say to create a separate filesystem
> for
> each user home directory. My instinct is to ask "Are you serious???".
> My
> gut feeling isn't entirely logical but the idea of getting 1000+ lines
> of output from a simple "df" just feels wrong...
> 
> Can anyone comment about how well this approach actually works,
> specifically on FreeBSD? (we're running 9.1) Obviously it has some
> nice
> features, such as quota controls, snapshots directly available to
> users
> within their home, etc, but it leaves me concerned. I chatted with
> some
> neighbors who have a larger, Solaris-based shop, and they said that
> with
> 10,000 user home filesystems, their server could take an hour to boot
> (at least using the default startup scripts). They reverted to having
> one big shared filesystem for all, but would like to revisit the
> per-user approach with fewer users per server.
> 
> Ours wouldn't be so large, but we could easily have around 1000 user
> filesystems. I haven't tested yet what effect that would have on boot
> time, though hope to test it over the next week. Perhaps it implies
> other resource usage besides the boot time issue (is there any limit
> to
> number of filesystems mounted or NFS-exported?). I wonder if anyone
> here
> has built a system along these lines and has experiences to share.
> 
Well, there isn't any limit to the # of exported file systems afaik,
but updating a large /etc/exports file takes quite a bit of time and
when you use mountd (the default) for this, you can have problems.
(You either have a period of time when no client can get response
 from the server or a period of time when I/O fails because the
 file system isn't re-exported yet.)

If you choose this approach, you should look seriously at using
nfse (on sourceforge) instead of mountd.

You might also want to contact Garrett Wollman w.r.t. the NFS
server patch(es) and setup he is using, since he has been
working through performance issues (relatively successfully
now, as I understand) for a fairly large NFS/ZFS server.
You should be able to find a thread discussing this on
freebsd-fs or freebsd-current.

rick

> Thanks for any comments,
> 
> Graham
> --
> -------------------------------------------------------------------------
> Graham Allan
> School of Physics and Astronomy - University of Minnesota
> -------------------------------------------------------------------------
> _______________________________________________
> freebsd-fs at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-fs
> To unsubscribe, send any mail to "freebsd-fs-unsubscribe at freebsd.org"