Thinking of using ZFS/FBSD for a backup system
Jeremy Chadwick
koitsu at FreeBSD.org
Wed Jul 9 05:56:46 UTC 2008
On Wed, Jul 09, 2008 at 07:44:20AM +0200, CZUCZY Gergely wrote:
> On Wed, 09 Jul 2008 01:14:49 +0200
> Kris Kennaway <kris at FreeBSD.org> wrote:
>
> > CZUCZY Gergely wrote:
> > I don't know; empirically my setup is an upper bound. How large was
> > "as large as it was allowed" for you?
> Well, we cannot buy "upper bounds" all over, just because some
> developer is unable to figure out things. I think you can't expect
> FreeBSD users to spend as much money as possible, just because the devs
> can't tell how much is enough...
> It seems more like a twilight zone then a stable feature now ;)
>
> It was exactly as much as an amd64 installation would allow with 2GB of
> physical memory. We've dismissed the setup around february, and I don't
> have the configs anymore. It was an amd64 setup with 2GB of physical
> memory.
The bottom line here is that i386 and amd64 both have a kmem_size limit
of 2GB. You can throw 32GB of RAM into an amd64 box, but FreeBSD will
only utilise up to 2GB of that for kmem. That is purely a FreeBSD
limitation, and is being dealt with in HEAD by Alan Cox. I believe he
has a patch, or it may have been committed -- I don't follow HEAD. I
can point people to a mailing list URL, if needed.
This is one of the limitations Gergely is referring to.
Since ZFS is incredibly memory-hungry, you're forced to tune ZFS to try
and get it to "play nice" with that 2GB limit on STABLE/RELEASE systems.
You also need to keep in mind that you can't just set kmem_size and
kmem_size_max to 2048M, because the kernel needs memory for other
things.
The tuning parameters I use on my 2GB amd64 and 4GB amd64 boxes are:
vm.kmem_size="1536M"
vm.kmem_size_max="1536M"
vfs.zfs.arc_min="16M"
vfs.zfs.arc_max="64M"
If you set kmem_size and kmem_size_max any higher than that, the machine
will panic on boot, stating (indirectly) that there isn't enough memory
available for the kernel to allocate for other things.
Until I added the arc_min and arc_max setting, I could occasionally
panic the machines under very heavy load (heavy zpool I/O), caused by
kmem exhaustion. Since adding the arc_* tunings, I've tried very hard
to crash the machines, and I cannot.
But there's absolutely no guarantee those tuning parameters above will
ensure FreeBSD won't panic due to kmem exhaustion. I believe this is
the point Gergely is making about the "stability" of the whole thing.
Now, with regards to prefetch_disable, folks can disable that if they
want. I disable it on my above systems because for what they do, the
overall performance appears better with prefetching disabled.
I hope this helps shed some light here...
--
| Jeremy Chadwick jdc at parodius.com |
| Parodius Networking http://www.parodius.com/ |
| UNIX Systems Administrator Mountain View, CA, USA |
| Making life hard for others since 1977. PGP: 4BD6C0CB |
More information about the freebsd-fs
mailing list