How many ram...

[Freddie Cash]

On Wed, Jan 14, 2015 at 9:00 AM, Albert Shih <Albert.Shih at obspm.fr> wrote:

>  Le 14/01/2015 à 10:50:18-0600, Linda Kateley a écrit
> > I will say one more thing.. I also have a customer who uses zfs for
> > security camera storage. The cameras deliver 100's of k bytes per
> > minute... But they save the data for a very very very long time. That
> > kind of system would need very little ram(maybe 8GB) but lots and lots
> > of disk.
>
> Thanks you very much.
>
> May I ask you something (feel free to not answer of course ). I saw on your
>
> > >> 32GB ram. http://kateleyco.com/?p=815
>
> you have install I quote
>
>   «hey will have 252 4TB drives in 6 45-drive chassis with multiple
>   controllers»
>
> do you have any idea how many pool they have ? how many disk they put in
> one raid ? how many raid they put in one pool ?
>
> Actually I've one server a very big pool (I known some tell me it's too
> big) with 72 disks in 6 raidz2.
>
> ​We have two storage systems with 90 harddrives each (2 TB drives).  These
are setup with 2 45-drive 4U JBOD chassis each, and a 2U head unit with
SSDs for the OS and L2ARC/ZIL, and the SATA controllers.  The way the
hardware is configured, they can each handle another 2 JBOD chassis without
daisy-chaining anything.

​Using 6-disk raidz2 vdevs, for a total of 15 raidz2 vdevs per storage
pool.​

​These are backup systems and an off-site replication system, so storage
throughput and IOps wasn't super critical, while storage space and
manageability were.  They only have gigabit NICs, and can saturate those
while running backups or "zfs send" / "zfs recv".

They do have 128 GB of RAM, though, more as a "get it now while it's cheap
instead of waiting until we need it" than anything.  And they have dual
8-core AMD Opteron CPUs (again, more as a "they're inexpensive now, so get
as much as we can afford" than any real need for it).

One of them has dedupe enabled (yeah, yeah, we know, we're moving away from
it, it's actually the last one with it enabled), and actually does use the
RAM for DDT storage in the ARC.  The other one doesn't have dedupe enable,
and most of the RAM sits "idle". ​

-- 
Freddie Cash
fjwcash at gmail.com