Re: TarFS

On Mon, Mar 25, 2024 at 9:59 AM Dag-Erling Smørgrav <des@freebsd.org> wrote:

> Mark Saad <nonesuch@longcount.org> writes:
> > I was wondering if anyone has started to play with tarfs in FreeBSD 14?
>
> I wrote it, does that count?
>
> > It appears to puke with larger tarballs.
>
> On the contrary, one of the use cases I tested was buildworld from a
> tarball of the source tree, which it handled just fine.
>
>
Dag let me just say , great work. I really like this.


> > For example, if I fetch a copy of ports or pkgsrc and decompress it to
> > just a posix tar archive I can't get it to mount.
> >
> > root@mono:/home/nonesuch # file ports.tar
> > ports.tar: POSIX tar archive
> > root@mono:/home/nonesuch # mount -t tarfs ./ports.tar /tarfs/ports
> > mount: ./ports.tar: Inappropriate file type or format
>
> % fetch ftp://ftp.freebsd.org/pub/FreeBSD/ports/ports/ports.tar.zst
> ports.tar.zst                                           47 MB 5431 kBps
> 09s
> % sudo mount -rt tarfs $PWD/ports.tar.zst /mnt
> mount: /home/des/ports.tar.zst: Inappropriate file type or format
> % dmesg | grep tarfs
> tarfs_alloc_one: unsupported global extended header at 0
> % zcat ports.tar.zst | hexdump -C | head -3
> 00000000  70 61 78 5f 67 6c 6f 62  61 6c 5f 68 65 61 64 65
> |pax_global_heade|
> 00000010  72 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
> |r...............|
> 00000020  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
> |................|
>
> This is a non-standard extension header emitted by `git archive` which
> contains metadata about the git tree from which the archive was created.
> I believe you can safely strip it off (zcat ports.tar.zst | dd bs=1k
> skip=1 >ports.tar), but you'll run into issues with very long paths in
> devel/electron*.  I will take a closer look when I find the time.
>
> Note that tarfs can mount tarballs compressed with zstd, but performance
> will be poor unless you create a multi-frame archive (try using --zstd
> --options=zstd:frame-per-file,zstd:min-frame-size=65536).
>
> DES
>

So I am not sure exactly what tarfs is yelling about with pkgsrc.tar , but
I wonder if it's a by-product of netbsd using pax .
NetBSD tar's dont work , OpenBSD stuff does and they use a ustar format.

root@mono:/home/nonesuch # cat pkgsrc.tar |hexdump -C |head -3
00000000  70 6b 67 73 72 63 2f 00  00 00 00 00 00 00 00 00
 |pkgsrc/.........|
00000010  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
 |................|
*
root@mono:/home/nonesuch # cat ports.tar  |hexdump -C |head -3
00000000  70 61 78 5f 67 6c 6f 62  61 6c 5f 68 65 61 64 65
 |pax_global_heade|
00000010  72 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
 |r...............|
00000020  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
 |................|


root@mono:/home/nonesuch # cat xenocara.tar   |hexdump -C |head -3
00000000  2e 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
 |................|
00000010  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
 |................|
*


Two other questions ( I have not tried to read the code , please forgive my
assumptions )  behind the scenes are the tarballs extracted over some fs,
or mounted directly and read ?
Second are there any contradictions with tarfs ; can they be nfs exported,
mounted on a tmpfs etc .

Again good work. C an we expect to see a libarchive-fs next :)


> --
> Dag-Erling Smørgrav - des@FreeBSD.org
>


-- 
mark saad | nonesuch@longcount.org