Re: support for pNFS with Linux as Data Servers

On Sat, May 10, 2025 at 8:26 PM David Chen <david.chen@peakaio.com> wrote:
>
> Hello!
>
> Currently, FreeBSD supports pNFS with File and Flexible File
> layouts. For the Flexible File layout, FreeBSD supports a tightly
> coupled locking model that requires FreeBSD servers as Data Servers
> (DSs). I'm interested in adding support for a loosely coupled locking
> model that would allow Linux machines to be used as Data Servers. To
> that end, I'd like to gauge interest and get feedback about that.
>
> AFAICT there are two changes needed to add this support. The first
> change is to allow for file handles of variable length and up to the
> maximum size allowed by the RFC. This seems relatively
> straightforward, and could be done by changing a bunch of uses of
> fhandle_t to instead use a data structure that can store larger file
> handles. Also, nfsrv_layoutget() and the use of NFSX_V4FLEXLAYOUT
> would need to be updated to allow for variable lengths.
>
> The second change is to support a loosely coupled locking
> model. Besides trivially setting the ffdv_tightly_coupled flag of
> ff_device_versions4 to false, I see two immediate issues. One issue is
> the FreeBSD's special 0x555555555555555555555555 state ID (along with
> its special seq num 0xffffffff) is, naturally, not understood by a
> Linux Data Server. The other issue is clients will use the synthetic
> uid/gid given by the MDS (currently 999/999), and this results in
> access errors when the clients talk to the DSs.
I think the hard part may be implementation of the "fencing" that is
required for the loosely coupled model. (For what is there now,
permission handling is done by the DS(s), since the files on the DS(s)
have exactly the same owner/group/mode/ACL as the MDS.)

It has been a while since I read the RFC, so I cannot recall how the
loosely coupled model supports permissions (fencing off files that the
client does not have permission to access).

I will be curious to hear if anyone else is interested, since I don't
think anyone actually uses what is there now?
(The mirrored case definitely needs work. It currently fails a DS
too easily.)

rick

>
> I've made some of these changes in a rough manner, not in a
> production-ready way, and I/O seems to work and some basic tests pass.
>
> Is the FreeBSD community interested in development in this direction,
> i.e. pNFS using FreeBSD for MDS and Linux for DSs?
>
> Thanks!
>