Thoughts on Multi-Symlink Concept

[Bakul Shah]

On Feb 12, 2014, at 9:08 PM, Jordan Hubbard <jordan.hubbard at gmail.com> wrote:

> 
> On Feb 11, 2014, at 9:09 PM, Jason Hellenthal <jhellenthal at dataix.net> wrote:
> 
>> Instead of: cat /path/to/files*
>> ln -sm /path/to/files* ./my_concat_list
>> cat ./my_concat_list
>> 
>> Or
>> 
>> ln -sm  /path/to/file1  /path/to/file2 ./my_concat_filters
>> pfctl -v -f ./my_concat_filters
> 
> Globbing is done in user land (by the shell) - you wouldn’t want to push that down into the kernel, which is either what you’d have to do or you’d need a user land daemon which did round-trips with the kernel to do the translation, which would also need to make sure to get all of the process permission stuff right since the user id / gid / $CWD would all potentially affect the expansion of the “symlink”.
> 
> There are other problems.  Since “my_concat_list” now expands to multiple files, you have to pre-expand its contents before actually execing cat(1), and since filenames can have spaces and other magic characters (to the shell) in them, you would need to be able to deal with escaping all of that.  Just like…  the shell does!
> 
> Even worse still, you’d be essentially creating a new file type with very non-file like semantics.  Even variant symlinks (/bin -> /${ARCH}/bin), which can expand differently depending on the user context, have clearly understandable semantics - you know that the symlink is going to expand to exactly one file no matter what ARCH is set to.  How, conversely, would you expect “cat < my_concat_filters” or “dd if=my_concat_filters of=…” to work?  You’ve essentially created a construct that can cause truly weird behaviors unless you use it in exactly one way and one way only (as a file argument list), the user having to explicitly know the difference when they consume the symlink.   That creates a whole raft of layering violations, and is very “not Unix”, to be honest.

May be a similar idea can be used to implement file striping & mirroring on top of existing filesystems -- modern distributed FS seem to reinvent a lot stuff. Instead one can just build a simple thin layer and use local and remote simple filesystems to create a distributed FS. I wouldn't use symlinks but indicate additional semantics via some metadata.

> If you want to get creative along these lines without breaking Unix, I think creating some new type of artificial directory that doesn’t actually physically contain files but can flexibly reference a collection of files based on some sort of query - that might be interesting.

Lot of such variations are possible using fusefs