Storage cluster advise, anybody?

Valeri Galtsev galtsev at
Fri Apr 22 21:57:40 UTC 2016

On Fri, April 22, 2016 2:38 pm, Mehmet Erol Sanliturk wrote:
> On Fri, Apr 22, 2016 at 12:17 PM, Valeri Galtsev
> <galtsev at>
> wrote:
>> Dear Experts,
>> I would like to ask everybody: what would you advise to use as a
>> cluster, or as a distributed filesystem.


>> <rant>
>> I really didn't mean to write this, but I figure it probably will
>> once I start getting your advices, so here it is. I did my research
>> my requirements in mind and came up with the solution: moosefs. It is
>> reviewed much, no reviews with criticism at all, and not much you can
>> could" I should say) find howtos about customizations, performance
>> etc. It installs without a hitch. It runs well, until you start stress
writing a lot to it in parallel, then it started performing
>> exponentially
>> badly for me. Here is where extensive attempts to find performance
>> documentation faces lack of success. What made my decision to never
>> use it in a future was the following. I started migrating data back
>> moosefs to local UFS (that is FreeBSD box) filesystem using rsync
>> What I observed was: source files after they have been touched by rsync
changed their timestamps. As if instead of creation timestamp it is an
access timestamp on moosefs. This renders rsync from moosefs useless, as
>> you can not re-run failed rsync, and you obliterate some of metadata of
the source ("creation" timestamp). I wrote e-mail to sourceforge
>> mail list, mentioning all this and the fact that I am using open source
moosefs. Next day they replied asking whether I use version 3."this" or
version 3."that", as they want to know in which of them they have a bug.
>> Whereas latest open source version they have everywhere, including
sourceforge is older version: 2.0.88.
>> Basically, my decision was made. Sorry for venting it out here, but I
figured, it will happen some moment when I will get your advises.
>> Thanks a lot for all your advises!
>> Valeri
> In page :
>  there are the following lines with download links :
> Current release version 3.0.73-1 (GPLv2) 2016-03-04
> Current release version 3.0.69-1 (GPLv2) 2016-01-12
> Current release version 3.0.55-1 (GPLv2) 2015-10-21
> Current release version 3.0.37-1 (GPLv2) 2015-05-15
> Current release version 3.0.17-1 (GPLv2) 2015-04-21
>  .
>  .
>  .
> Perhaps they assumed you are using one of them .

Thanks for pointing that out. I see now that they are there under the name
"current" as opposed to 2.0.88 "stable" - pretty much consistent with what
FreeBSD terminology would be. So, 2.0.88 _is_ the latest "stable".

No, I'm using "stable" on all production everything (my laptop doesn't
count). And the weirdness I see is in production version (built on my
machine from port I must admit).



> Mehmet Erol Sanliturk

Valeri Galtsev
Sr System Administrator
Department of Astronomy and Astrophysics
Kavli Institute for Cosmological Physics
University of Chicago
Phone: 773-702-4247

More information about the freebsd-questions mailing list