HTTPS on freebsd.org, git, reproducible builds

[Polytropon]

On Sat, 19 Sep 2015 20:28:39 +0300, Slawa Olhovchenkov wrote:
> On Sat, Sep 19, 2015 at 06:47:12PM +0200, Polytropon wrote:
> 
> > > > As far as I could understand, pkg will deal with the components
> > > > comprising the OS in the same manner as it does for the ports
> > > > collection. So the kernel, the userland, the sources and so on
> > > > will "become packages" for pkg to install or upgrade. This is
> > > > a similar approach to common package management on Linux, except
> > > > that Linux (as a term to summarize all the many distributions)
> > > > doesn't have an OS ("the base OS") per se.
> > > 
> > > This is very bad.
> > 
> > Don't worry. The OS will still be maintained by the FreeBSD team.
> > And the components which the OS is composed of will probably not
> > be separated into hundreds of separate packages (as it is in
> > Linux - where the distribution creators decide which packages
> > belong to a base install, like, which package installer, which
> > shell, X or no X, and so on).
> > 
> > In the end, it might look like there are few additional packages
> > that will be installed: sys_bin, sys_src, sys_ports and so on.
> > An update you perform with freebsd-update will then be an update
> > on the sys_* packages with pkg, leading to a binarily upgraded
> > operating system. You then _can_ upgrade your ports collection,
> > or you can leave it as is. This is the advantage of FreeBSD:
> > The OS and the additionally installed (3rd party) software are
> > beging dealt with independently.
> > 
> > And this is good. :-)
> 
> I am don't see advantage of this.
> What's part of systeam I am don't need to install?

The components won't be that separated. No direct "part of
the system" will exist, like, "do I install sh, or can I
live without it?"; I'd rather assume that there are only
few packages that result in a fully functional (!) operating
system. Still I hope the pkg approach will give you the
flexibility of src.conf - to omit components you _really_
don't need, and where you _intend_ to leave them out.



> ports?

You don't need the ports tree installed to get the OS running.



> this is don't need released as package, when I need /usr/ports
> I am need it from svn (not from portsnap or else).

Installing the ports tree via pkg is the same as installing
the port tree via ports.txz - of course at the relase date.
Subsequent additions can be made with svn or portsnap (binary
upgrades to ports tree - this is what a pkg upgrade of the
ports tree would probably look like).



> src? also svn.

When you simply want the RELEASE sources, installing svn and
having it run is probably more work than simply downloading
src.txz and uncompressing it into /usr/src - again, this is
what pkg would do.



> separately userland parts? I am can't imagine how to install Kerberos
> separately. many other userland parts tightly intergrated together.

The ports won't "fall apart", and they won't integrate much
closer with the OS than they currently do.



> Yes, I can build custom system with off some parts in src.conf, but
> this system will be very special and need some knowelege.

Exactly. That's why using pkg to install and upgrade the OS
won't significantly change the way you install things.



> > > > You can already see this kind of development: The documentation
> > > > has become a package, and the package manager itself is a
> > > > package (separated from the OS, which only contains a bootstrap
> > > > loader for the real program). Finally, the installation process
> > > > could become a task of "pkg install", instead of "tar xf". And
> > > > a unification of the infrastructures could lead to additional
> > > > benefits (only _one_ system for both components - OS and ports).
> > > 
> > > I am have many troubles with this way in Linux.
> > > Kernel and userland versions mismatch.
> > > glibc version incompatible with rpm.
> > > pkunzip.zip problem.
> > > And etc.
> > 
> > I know what you're refering to. :-)
> > 
> > On Linux, an "upgrade everything" process might involve a kernel
> > or a system library update not properly being dealt with in
> > "userland" (if I may abuse the term in this context). Now you
> > have a system that won't boot anymore, and you might not even
> > be able to reach a kind of maintenance mode (like FreeBSD's
> > single-user mode with /rescue) because somehow your fallback
> > kernel and libraries got deleted...
> > 
> > Of course FreeBSD also can run into this kind of problem, but
> > the OS is always consistent. An upgrade does _not_ break the
> > OS. It _might_ break ports. During the course of -STABLE, this
> > usually does not happen (because the interfaces are stable).
> > That's why you always see the advice to recompile (or reinstall)
> > your ports when you move to a new major version, leaving the
> > path of -STABLE.
> 
> From last: pkg (utility `pkg), building on 10.2 can't be run on 10.1
> because used newer symbols from libc.so. Now imagine system long time
> not updated. EoL is come in. How I upgrade this? pkg want to upgrade
> himself, fresh version for outdated system don't exist, new version
> can't be run... Deadlock?

I currently have a similar system here. No further updates
possible, ports infrastructure has changed too much. Only
solution: New installation. :-)

The OS's pkg binary is just a bootstrap loader for the real
one installed as a package. It's possible that the same
approach will be kept when pkg manages the OS components.



> Next, how to upgrade system? kernel first? ok. for this case kernel
> can't be depend from userland packages. How to upgrade to
> correspondend userland packages?

I'd say that a "pkg upgrade" of the userland and the kernel
have to go hand in hand, as it is suggested today, because
kernel and world have to be in sync. The operation will be
similar to what you do today with "freebsd-update upgrade".
Of course this requires a good coupling between the pkg port
and the (updated) OS.



> And we can got network unreachable
> system (I am remember time ifconfing interface change).

No difference to how it's handled today. First, everything
needed is downloaded, then the upgrade process starts,
probably keeping a "fallback" solution available.

If you're worried here, you should have a look at Boot
Environments (as known from Solaris): FreeBSD + ZFS + beadm
is a very good solution for preparing, testing, and maybe
rolling back upgrades.



> What about -current?

The -CURRENT (or -HEAD) development branch will surely not
be available via pkg upgrades. They are, as today, done from
the source.



> userland first? ok. Got new libs with missing syscals and we can't run
> any program.

Dynamic linking to the system's most essential library should
not break things. Stable interfaces are very important here,
so the upgrader won't be so stupid to shoot his own foot. :-)



> Now about embeded systems.
> 
> -rw-r--r--  1 root  wheel   2.4M Jul 27 01:16  /var/cache/pkg/pkg-1.5.5-20bbe78419.txz
> # ls -l /var/db/pkg/
> total 2206
> -rw-r--r--  1 root  wheel      246 Aug  3 16:41 ivs.meta
> -rw-r--r--  1 root  wheel  1384448 Aug  5 22:31 local.sqlite
> -rw-r--r--  1 root  wheel   342016 Aug  3 16:41 repo-ivs.sqlite
> -r--r--r--  1 root  wheel  3804129 Sep 18 04:03 vuln.xml
> # du -hc `pkg info -l pkg`
> 6.1M    total
> 
> I.e. package management overhead about 11MB. Or I missing somewhere?
> Oh, I am need double space for system: .txz and expanded.
> 
> And what advantage for this?

The pkg creators chose to transform the /var/db/pkg subtree
and the text files into databases. The system provides libraries
to query them. On my old FreeBSD 8 home system, /var/db/pkg
is 44M in size.


Please keep in mind that I'm just mentioning my own thoughts
here. I'm not part of the pkg development team. If you have
specific questions regarding the use and implementation of
the upcoming OS updating mechanism, you should contact the
designated maintainers.



-- 
Polytropon
Magdeburg, Germany
Happy FreeBSD user since 4.0
Andra moi ennepe, Mousa, ...