[Bug 196273] New: truncate(1) adding posix_fallocate(2) support to /usr/bin/truncate -- diff with unit tests attached

[bugzilla-noreply at freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=196273

            Bug ID: 196273
           Summary: truncate(1) adding posix_fallocate(2) support to
                    /usr/bin/truncate -- diff with unit tests attached
           Product: Base System
           Version: 10.1-STABLE
          Hardware: Any
                OS: Any
            Status: New
          Severity: Affects Only Me
          Priority: ---
         Component: bin
          Assignee: freebsd-bugs at FreeBSD.org
          Reporter: bugmeister at ba23.org

Created attachment 150961
  --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=150961&action=edit
patch -- you need to run one svn cp before you run patch

RFC: adding posix_fallocate(2) support to /usr/bin/truncate

TL;DR
=====
I have provided a patch that gives minimal posix_fallocate(2) support to
/usr/bin/truncate. Now truncate(1) gives the user a choice create a sparse file
or to create a pre-allocated space on disk.

Before you apply the patch, you need to svn cp an existing file:
svn cp usr.bin/truncate/tests/truncate_test.sh
usr.bin/truncate/tests/base_test.func


Problem to Solve
================
The truncate utility is often used to create large files used as disk images by
emulators or as the backing storage for memory disks. One issue is that a
sparse file is created -- the actual disk space is not allocated until you
write. Some users might prefer to get the out-of-space errors when you
initially create the large file, instead of getting out-of-space errors during
future writes.

The man page for truncate(1) suggests using dd(2) to create a pre-allocate
space. The idiom is to use dd(2) to read data from /dev/zero and write zeroes
to the file, causing the space to be allocated on disk. With the existence of
the new kernel call posix_fallocate(2), it seems odd to writes zeroes to
pre-allocate space. It might be useful to have a command line utility call
posix_fallocate(2) as an alternative to the write zeroes idiom. Today, there is
no FreeBSD command line utility that uses posix_fallocate(2).

Proposed Solution
=================
Add a new command line flag "-a" to the truncate(1) command line utility. When
"-a" is present, truncate(1) will use posix_fallocate(2) when increasing the
size of a file.

Examples
========

rm -f afile
truncate -s 1m afile

Creates a 1048576 byte file -- truncate(2) will be used so the file is expected
to be a sparse file -- not all the 1048576 bytes will be allocated to disk.

rm -f afile
truncate -a -s 1m afile

Creates a 1048576 byte file -- posix_fallocate(2) will be used so all the
1048576 bytes will be allocated to disk -- no holes are expected to be present.

rm -f afile
truncate -s 1m afile
truncate -a -s +1m afile

Creates a 2097152 byte file -- truncate(2) will be used for the first 1048576
bytes and   posix_fallocate(2) will be used for the last 1048576 bytes. The
last 1048576 bytes will be allocated to disk.  At least one hole is expected to
be present at the start of this file.

Proposed Test Strategy
======================

The existing truncate_test.sh tests can be run twice -- once with the proposed
"-a" command line flag and once without the new flag. Ignoring the allocation
of disk space, the behaviour of the truncate(1) utility should be the same in
both cases.  This will require the existing tests to be moved into a shared
file used by the existing truncate_test.sh tests and a new script that will use
the proposed "-a" command line flag.

A new test needs to be written that will detect if the new file created by
truncate(1) is sparse or not.  This test will be written in C and use
lseek(2)'s SEEK_HOLE feature to count the number of sparse bytes in a file.  It
is expected that posix_fallocate(2) will create files with a sparse byte count
of zero, while truncate(2) will create files with a sparse byte count that is
greater than zero.

Rejected Solutions
==================
1) GNU coreutils provides both truncate and fallocate command line utilities.
The GNU coreutils fallocate command line utility that will use
posix_fallocate(2) for creating pre-allocated files. The FreeBSD truncate(1)
utility's command line args are compatible with the GNU coreutils truncate
utility -- both utilities create sparse files. FreeBSD does not have a
fallocate utility. It was assumed that adding a new command line utility to
FreeBSD would not be plausible. This proposal adds minimal posix_fallocate(2)
support to the existing truncate(1) utility, instead of adding another command
line utility.  This proposed "-a" FreeBSD extension is not command line
compatible with GNU coreutils truncate or fallocate. 

2) It should be possible to use posix_fallocate(2) to target an existing sparse
file. This should replace the existing holes with equivalently-zeroed allocated
disk space. It seemed odd to add this feature to truncate(1).

See Also
========
posix_fallocate(2), truncate(2), truncate(1), mdconfig(8), dd(2), lseek(2)

-- 
You are receiving this mail because:
You are the assignee for the bug.