Distributed, snapshotting, checksumming filesystems for FreeBSD

[Benjamin Kaduk]

On Tue, 28 Feb 2012, Lawrence Stewart wrote:

> On 02/25/12 15:37, Kamil Choudhury wrote:
>> The dream: a file system spread out over a variable,
>> ever increasing number of hosts, presenting a single
>> unified file system to any client host mounting the
>> file system.
>> 
>>> From the client's point of view, it is possible to
>> snapshot the directory view that is presented. The
>> client also has confidence that data written to the
>> file system will be returned exactly as it went in.
>> 
>> Now that I think about it, what I seem to be looking
>> for is a network aware ZFS that uses hosts as vdevs.
>> 
>> Is there such a thing out there?
>
> AFS perhaps?
>
> http://en.wikipedia.org/wiki/OpenAFS
> http://en.wikipedia.org/wiki/Andrew_File_System
>
> /usr/ports/net/openafs

I almost made that suggestion, but decided not to, given that the present 
implementation only allows administrators to make snapshots, and I don't 
think any checksumming would be quite what was described.  (One could 
certainly use a ZFS partition for the fileserver backing store, which 
would give some checksum functionality, but it would not be exposed to 
users in a particularly useful fashion.)  But, it does win at having a 
single unified file system presented to any client mounting it, which is 
distributed over multiple servers (and data can be shuffled between 
servers seamlessly).
If the OP can compromise in the right directions it might be worth looking 
at, though, I suppose.

I'll also add that the OpenAFS client is not production-ready on FreeBSD; 
there are some memory leaks and issues with cache eviction for large files 
that are not yet resolved.  Unfortunately there have been a couple of 
regressions on the git master branch that I need to track down before I 
can get back to working on the FreeBSD-specific issues.

-Ben Kaduk