RDBMS features into VFS
Kyryll A Mirnenko aka Mirya
mirya at matrix.ua
Sun Feb 19 05:53:30 PST 2006
Consider this a feature request.
THE PROBLEM
The userland is missing "ORDER BY" and "LIMIT"-like features for directory
reading. E.g. The only avaliable API is readdir() interating over the
directory in unspecified order. On other hand most filebrowsers utilize file
sorting extensively (mc, for, ex, will sort files by name as default option).
When working with large directories this slowdowns operation much - e.g.
the filebrowser must read huge filelist, then sort it and output only a small
part of it - despite the fact some FS have BTREE index for filenames. Other
potentional candidates for sorting are:
- mtime
- size
- extattrs
THE SOLUTION
Add an api into VFS layer allowing to fetch a sorted filelist and limiting the
output to a range from N to M, like it does "SELECT ... FROM ... ORDER BY
field LIMIT a, b" in MySQL notation. Underlying FS'es are allowed not to
add any optimized API for this operation, if so VFS implements it by itself
THE USE
Someone may criticise the exaple above as huge directories in REAL fs are
exception, but the main application comes when talking about syntetic FS.
After fusefs was ported to FreeBSD it became possible to represent some
business logic as filesystem and "merge" it into the real one w/o digging in
the kernel and thinking much about how a bug in your code can make the whole
system crash. FS interface allows programmers not to care much about the user
iface as the user is free to use any HTML editor he/she prefers to edit a
piece of data provided by an application - reading and writing will be
forwardrd to aprop. backend via fuse.
The issue is most virtual "directories" in such systems will contain hundred
thousand of "files" (like customers table) so while underlying RDBMS can
effectively sort (the fields not representing standart file attributes are
mapped to extattr) and chunk the data, the fs layer is not aware about that.
--
Regards, Mirya
ICQ #313898202
More information about the freebsd-fs
mailing list