Converting sysinstalled FreeBSD into ZFS-only server.

[Andriy Bakay]

Pawel Jakub Dawidek wrote:
> On Mon, Aug 09, 2010 at 04:22:42PM -0400, Andriy Bakay wrote:
>> 1. In your post you are using dedicated partition for swap. Did it provide
>> any advantages versus swap on ZFS volume?
> 
> There is a good chance to deadlock your system when you have swap based
> on ZVOL, as to operat on ZVOL ZFS needs to allocate some memory, which
> you probably don't have much when you're swapping.
> 

Just to clarify. It means I can only use swap based on ZVOL only in 
non-production environment?

Could this problem be solved or it is something fundamental (for 
FreeBSD) here? As far as I know on Solaris ZVOL based swap works without 
problems.

>> 2. You are suggesting to set 'vm_kmem_size' value to 150% of RAM. What
>> pros. and cons. against following formula:
>>
>> vm_kmem_size = RAM / 2
>> vfs.zfs.arc_max = vm_kmem_size - 512M
>
> Well, I prefer to have as much address space as I have RAM, when you
> count VM fragmentation in, 150% should be enough to be able to allocate
> even entire memory for kernel. Of course it all depends on what you run,
> etc. This is setting I use and I had no problems with it.
>

I have FreeBSD 8.0-RELEASE-p2 amd64 with 4G RAM, when I set vm_kmem_size 
to 6G my vfs.zfs.arc_max became 5G. Is it normal for this kind of setup? 
Could such ARC value become a problem, because I only have 4G RAM?

>> 3. Could you be more verbose about your ZFS layout, what major advantages
>> it provide against for example the following:
>> http://wiki.freebsd.org/RootOnZFS/GPTZFSBoot. I found it very interesting
>> but I need more info.
> 
> I added some notes why I use and recommend such layout at the end of the
> post. From my experience the fact that file systems are cheap in ZFS
> doesn't mean it is to have too many of them. For example I don't like to
> have all /var/ subdirectories are separate ZFS datasets. There are
> strong reasons to separate some of them (to turn on compression, for
> example), but not all of them.
> 
> Of course if you don't like my reasoning or you have different needs,
> feel free to use whatever layout you feel fits best for you:)
> 
> I'm also open to comments on the layout I proposed. I use it for quite a
> while now and I tried different ones before too, but this one I simply
> find the best. If our official installer will support creating ZFS-only
> install, I'll be forcing this layout, so if you think something is
> _very_ wrong about it, let me know.
> 

Your notes in the blog 
(http://blogs.freebsdish.org/pjd/2010/08/06/from-sysinstall-to-zfs-only-configuration/) 
sounds very reasonable. So, should /var/db be compressed as well? In 
your layout you create home dataset under /usr: system/usr/home. maybe 
it is more logical to create it one level upper: system/home or in this 
case you just follow standard FreeBSD filesystem layout?

Thanks for your answers.
Andriy