[Bug 265651] [NEW PORT] archivers/zpaqfranz: versioned/snapshot archive

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=265651

--- Comment #20 from Franco Corbelli <franco@francocorbelli.com> ---
>First of all, about the architecture restriction again. Of course, as you're also 
>the upstream author, you're in two different roles here. If you remove code you 
>consider "untrusted", so it will (for now) only work on amd64, that's fine. 
>(...)
>So, in a nutshell, are you really sure about that?

I asked a user for help to try the software on non-Intel little endian (ARM)
architectures and, as I expected, there was indeed a (minor) problem due to
different compiler standards, fixed, in UTF-8 handling (note: with "UTF-8" I
mean briefly the whole actual filename storage chain, I know "pretty" well what
UTF-8 is, semantics are looser here)

I then bought a Macintosh PowerPC, to do certification tests on BIG ENDIAN
machines (where I expect more problems, due the advanced hashers used)
I have to fix the relative cabling and then make it work (which is not trivial)
Work in progress

I then implemented a new self-test function (which does not exist in the
FreeBSD release) which, in theory, should allow to verify (most, certainly not
all) the main functions

Basically the program now contains an archive of about 200KB (which weights
down the executable but oh well), which is extracted, togheter with a special
folder structure that is then created with pseudorandom files and, finally, a
real script (.sh for non-Windows and .bat for Windows) is prepared by zpaqfranz
I'm working on it, it's not trivial

This will become something like
zpaqfranz autotest -to /tmp/testfolder
Then you run /tmp/testfolder/dotest.sh following the instructions.

>Testing the port, i noticed two more things:

>* You link statically. Normally, you only do that for distributing software 
>outside of any "package management", and FreeBSD ports would typically patch 
>upstream Makefiles not to do that if there's no really unavoidable reason. Now, 
>as you are the upstream author yourself, would you mind changing your own 
>Makefile so it *doesn't* link statically? (related question, why is C++ code 
>compiled using ${CC}, so you have to explicitly link libstdc++?)

The issue of static linking is very complex and articulated.
There are pros and cons to both choices.
Personally, I believe that in 2022 the costs (of non-static linking) are much
greater than the advantages.

zpaqfranz is a kind of "Docker" or "backup-busybox" (of course I'm using a
metaphor). 
In the very latest version I just implemented the equivalent of the pause
command of Windows scripts to use them in *nix ones.

Each platform has its own preferences: for example RedHat and "cousins" do not
like them for many reasons (you can see them in the official documentation).
Most of which are laughable, seems to be back to segments and overlays, when I
was writing the first programs in 1985

Similarly on archlinux, but for a different reason (the PIE and RELO
enforcing): the package-tester is "offended" and issues warnings,for no real
reason. 
But they are automatic tools, and nobody likes warnings
On MacOS binaries are practically impossible to link statically, Apple goes to
great lengths to prevent this

So why link statically ?
Because it works.
There are no "breaks" due to updates, upgrades or whatever, symlinks that
"magically" disappear, collisions with other software, with the operating
system or whatever

A kind of little "Docker", remember?

The software (of this kind) must works, and that's what counts the most,
basically in all circumstances.
I administer several servers, where it happens from time to time that updates
forced by external reasons (EOL) lead systems to "break" something.
It is not a huge problem, it can generally be corrected, perhaps by renting a
KVM IP for a day in the worst case.

But this "category" of software (not just zpaqfranz, I mean "backupper") are
different: typically used for backup (and restore) of systems in production,
where it is unthinkable to suspend backups, perhaps for days, because the X.Y
library for some reason no longer makes your crontab-copy procedure runs after
some kind of upgrade for the "magicalserver 3.1" to "magicalserver 3.1a".

In some case it is impossible to do a rollback (you lose client-uploaded data)
or are extremely expensive

Short version: virtually zero security issues (I see no need for constantly
updated libraries, it's NOT some kind of daemon interacting with internet that
can be targeted). 

Virtually zero need for updating (essentially read-write to disk, malloc() and
little more) so the very latest version of the library with new
superdupefunction is just useless

As mentioned it is a cost-benefit balance, there is no "perfect" solution

On systems where it is mandatory not to do static linking ... I don't.

If static linking is not allowed on FreeBSD, then let it be dynamic

>* The 'dir' incarnation vanished completely. 
>...please just add a symlink. 

It is a minor problem: if I manage to make this port finalized (zpaqfranz), I
will make another one, almost identical, where the executable will be called
just... dir :)

Why not a symlink ?
The symlink operates simul stabunt vel simul cadent, and that's NOT what I
want.

Because it happens to remove the executable of zpaqfranz, keeping the one of
dir.

Why?
Because is it normal to have DIFFERENT versions of zpaqfranz in different
directories, not just /usr/local/bin

Without of course jails or whatever (too complicated - too much at risk of
malfunctioning)

When there is a (backup) program update "you" can't just change the executable
and that's it, from version 5.3 to 5.4.

You will need to keep at least two different program versions (the old and the
new) and two different backup archives (the old and the new)
It is not uncommon to have even 4 different versions, when there are major
changes
Within, say, 6 months, when you have done numerous data restores with the new
program, then you will replace "the" /usr/local/bin executable.

In the meantime, you may erase (depends on the priority in the PATH of your
scripts) /usr/local/bin/zpaqfranz

While the "command" dir does not require these special precautions: if, for
some reason, it's buggy, never mind, you'll use ls and df

On the other hand, the occupation of space is irrelevant (~2MB)

Short version: this category of programs (archivers, backups, etc.) requires
special precautions and the modalities I adopt reflect what happens when using
them in practice
It may sound like paranoia.
But it is only experience

Thanks anyway for your interest

-- 
You are receiving this mail because:
You are the assignee for the bug.