Re: Sparse file support in FreeBSD NFSv4.2 server

On Tue, 13 May 2025 at 10:53, Rob Norris <robn@despairlabs.com> wrote:
>
> On Tue, 13 May 2025, at 6:35 PM, Aurélien Couderc wrote:
> > On Tue, May 13, 2025 at 8:51 AM Rob Norris <robn@despairlabs.com> wrote:
> > > Without having looked at it, I can see a way to do it by creating some object-specific operation to "write" but have it accounted to a dataset's ""reservation", rather than "used". Easy to say, difficult to do. I suspect the hardest part is figuring out the best way to keep a set of reserved ranges on each object.
> >
> > What about just writing 0x00 bytes in case of F_ALLOCSP?
>
> By default, with block compression enabled, OpenZFS will detect all-zero blocks and write a hole instead, which doesn't use any space.

A "sparse file hole"? That would BREAK databases and lots of
scientific software.
There is a distinctive difference between a "sparse file hole", which
represents a range of "no data here", and a data range, which might
contain lots of 0x00 bytes, but represent VALID DATA. That is a
difference!

For example, for our data runs we create a sparse file with 310PiB
(Pebibyte), 10 Pebibyte per day of the current month. If we collect
data, the data will be written, filling part of the hole. But if there
were no data collected, that part of that file will remain a hole, and
indicate "no data (collected)"

> But even if it did, that can't guarantee enough space to be able to overwrite it, because the previous data may exist in snapshots or clones. There's also metadata and indirect blocks and other stuff that we need space for too.

I think Aurélien was thinking about "sparse file" as use case here,
not "deduplication" or "safe erasure of data"

> Reservations already have all the stuff needed to track the extra commitment, which is why I think if it were possible, that's the way to do it.

F_ALLOCSP *is* a reservation

Lionel