zfs home directories best practice

[Mehmet Erol Sanliturk]

On Wed, Apr 3, 2013 at 4:56 PM, Rick Macklem <rmacklem at uoguelph.ca> wrote:

> Graham Allan wrote:
> > We're building a new NFS home directory server on FreeBSD with ZFS.
> > The
> > Solaris ZFS Best Practices docs say to create a separate filesystem
> > for
> > each user home directory. My instinct is to ask "Are you serious???".
> > My
> > gut feeling isn't entirely logical but the idea of getting 1000+ lines
> > of output from a simple "df" just feels wrong...
> >
> > Can anyone comment about how well this approach actually works,
> > specifically on FreeBSD? (we're running 9.1) Obviously it has some
> > nice
> > features, such as quota controls, snapshots directly available to
> > users
> > within their home, etc, but it leaves me concerned. I chatted with
> > some
> > neighbors who have a larger, Solaris-based shop, and they said that
> > with
> > 10,000 user home filesystems, their server could take an hour to boot
> > (at least using the default startup scripts). They reverted to having
> > one big shared filesystem for all, but would like to revisit the
> > per-user approach with fewer users per server.
> >
> > Ours wouldn't be so large, but we could easily have around 1000 user
> > filesystems. I haven't tested yet what effect that would have on boot
> > time, though hope to test it over the next week. Perhaps it implies
> > other resource usage besides the boot time issue (is there any limit
> > to
> > number of filesystems mounted or NFS-exported?). I wonder if anyone
> > here
> > has built a system along these lines and has experiences to share.
> >
> Well, there isn't any limit to the # of exported file systems afaik,
> but updating a large /etc/exports file takes quite a bit of time and
> when you use mountd (the default) for this, you can have problems.
> (You either have a period of time when no client can get response
>  from the server or a period of time when I/O fails because the
>  file system isn't re-exported yet.)
>
> If you choose this approach, you should look seriously at using
> nfse (on sourceforge) instead of mountd.
>
> You might also want to contact Garrett Wollman w.r.t. the NFS
> server patch(es) and setup he is using, since he has been
> working through performance issues (relatively successfully
> now, as I understand) for a fairly large NFS/ZFS server.
> You should be able to find a thread discussing this on
> freebsd-fs or freebsd-current.
>
> rick
>
> > Thanks for any comments,
> >
> > Graham
> > --
> > -------------------------------------------------------------------------
> > Graham Allan
> > School of Physics and Astronomy - University of Minnesota
> > -------------------------------------------------------------------------
> > _______________________________________________
> >
>


I am sorry that , my previous message sent early .

After trying some more retry , it worked :

http://sourceforge.net/projects/nfse/
(
NFSE for FreeBSD NFS server )


Thank you very much .

Mehmet Erol Sanliturk