HTTPS on freebsd.org, git, reproducible builds

[Polytropon]

On Sat, 19 Sep 2015 22:31:05 +0300, Slawa Olhovchenkov wrote:
> On Sat, Sep 19, 2015 at 08:47:45PM +0200, Polytropon wrote:
> 
> > > > In the end, it might look like there are few additional packages
> > > > that will be installed: sys_bin, sys_src, sys_ports and so on.
> > > > An update you perform with freebsd-update will then be an update
> > > > on the sys_* packages with pkg, leading to a binarily upgraded
> > > > operating system. You then _can_ upgrade your ports collection,
> > > > or you can leave it as is. This is the advantage of FreeBSD:
> > > > The OS and the additionally installed (3rd party) software are
> > > > beging dealt with independently.
> > > > 
> > > > And this is good. :-)
> > > 
> > > I am don't see advantage of this.
> > > What's part of systeam I am don't need to install?
> > 
> > The components won't be that separated. No direct "part of
> > the system" will exist, like, "do I install sh, or can I
> > live without it?"; I'd rather assume that there are only
> > few packages that result in a fully functional (!) operating
> > system. Still I hope the pkg approach will give you the
> > flexibility of src.conf - to omit components you _really_
> > don't need, and where you _intend_ to leave them out.
> 
> # man src.conf | col -b | grep -c WITH
> 227
> 
> and many items can't be do as packages.

Correct. I think the approach with pkg for the OS will be
the same as with most packages: A predefined set of options
will be the default, from which the packages are built. If
you _intend_ to use nonstandard options, use the source Luke
and compile the things yourself. You simply cannot maintain
2^227+ packages. :-)



> > > this is don't need released as package, when I need /usr/ports
> > > I am need it from svn (not from portsnap or else).
> > 
> > Installing the ports tree via pkg is the same as installing
> > the port tree via ports.txz - of course at the relase date.
> > Subsequent additions can be made with svn or portsnap (binary
> 
> you can't be update bu svn not svned ports tree.

It's recommended to obtain a "clean" ports tree (initial
checkout) when you want to use svn; if you're going to
stay with portsnap, you can do this from a RELEASE ports
tree version without problems.



> > upgrades to ports tree - this is what a pkg upgrade of the
> > ports tree would probably look like).
> 
> if you need ports tree -- most likely you need ports tree modification
> and this is disallow binary upgrades to ports tree.

For everything non-standard, poudriere is a nice tool.
Of course you can use "pkg lock" / "pkg unlock" if you
know what you're doing (and if it's just about less than
a handful of ports).



> > > src? also svn.
> > 
> > When you simply want the RELEASE sources, installing svn and
> > having it run is probably more work than simply downloading
> > src.txz and uncompressing it into /usr/src - again, this is
> > what pkg would do.
> 
> Many imporvement happened between releases.
> If you accpeted RELEASE -- you mostly accpeted GENERIC kernel and
> don't need source anyway.

Correct - except you _intendedly_ want a non-GENERIC kernel.
In this case, you use the source version you want, updated
the way you chose. Anyway, if this is a concern, you _know_
what you're doing. Dealing with sources, even though easy,
is nothing the typical "n00b" user will do. :-)



> You can do WITHOUT_KERBEROS and WITHOUT_KERBEROS_SUPPORT and totaly
> remove kerberos suuport as library, binary and hooks in ssh, sshd,
> login and etc. How do this by pkg?

Probably you _don't_ do this with pkg. As I said in comparison
to ports (or regarding the OS, in relation to freebsd-update),
you do this from source. Binary packages simply _cannot_ accomodate
to exponentionally growing amounts of variations of options set
or not set. :-)



> > > Yes, I can build custom system with off some parts in src.conf, but
> > > this system will be very special and need some knowelege.
> > 
> > Exactly. That's why using pkg to install and upgrade the OS
> > won't significantly change the way you install things.
> 
> Hmm. kernel+userland as mega-package? Or how?

Yes, I'd say that this is how it will be designed, compare
to kernel.txz + base.txz, as the installer (bsdinstall)
would use them. Of course it would also be possible to
use "sub-packages", each one representing a part of the
/usr/src subtree, but I assume this would be the wrong
way go - "dependency hell" preprogrammed. :-)



> Ports may be additional troubles (you software need php52, php52
> absent in modern ports tree, for example).

Right. Those are typical edge cases where the default
options and the precompiled binaries won't work. Just
imagine media software like mplayer: with all the codecs
(introduces new dependencies!) or only few ones, with
mencoder or not, with gmplayer/gmencoder GUI or not,
with skins or not... there cannot be packages for every
imaginable case.



> > The OS's pkg binary is just a bootstrap loader for the real
> > one installed as a package. It's possible that the same
> > approach will be kept when pkg manages the OS components.
> 
> And I am talk about imposible loading real one.

Without network connection - a big problem. But what's the
use of a binary upgrading tool without being able to reach
the remote source required? :-)

The idea of moving pkg out of the OS is - if I understood
the statements of its maintainers correctly - to allow
faster development and improvement of the pkg tool. It's
not directly tied to the system anymore (as the pkg_* tools
were), and because of that, it's a port, not an OS component.
The "not so fast changing" bootstrap loader therefore only
will get you "the real thing".



> > > Next, how to upgrade system? kernel first? ok. for this case kernel
> > > can't be depend from userland packages. How to upgrade to
> > > correspondend userland packages?
> > 
> > I'd say that a "pkg upgrade" of the userland and the kernel
> > have to go hand in hand, as it is suggested today, because
> > kernel and world have to be in sync. The operation will be
> > similar to what you do today with "freebsd-update upgrade".
> > Of course this requires a good coupling between the pkg port
> > and the (updated) OS.
> 
> Upgrading kernel and userland don't match pkg ideology, IMHO.

That's a possible and valid way to see things: pkg is the
successor of the old pkg_* tools, which dealt with ports,
not with the OS.



> > If you're worried here, you should have a look at Boot
> > Environments (as known from Solaris): FreeBSD + ZFS + beadm
> > is a very good solution for preparing, testing, and maybe
> > rolling back upgrades.
> 
> Not now. bsdinstall now use very bad partioning for BE.

When dealing with ZFS, I prefer not to deal with bsdinstall
and its "sane defaults"... it makes life easier. :-)



> > > What about -current?
> > 
> > The -CURRENT (or -HEAD) development branch will surely not
> > be available via pkg upgrades. They are, as today, done from
> > the source.
> 
> Shit, you talk about unification and SUDDENLY we got two, incopatible
> way -- for current and fro stable.

It has always been that way: -CURRENT (or -HEAD) is a development
branch. It's not stable. There are times where the -CURRENT
version crashes right away. Sometimes, it won't even build! Update
some hours later, and it works again. Experimental features may
be introduced in -CURRENT, and two weeks later, those features
have been removed. There sometimes are (binary) snapshots. Then,
-STABLE is the branch where "refinement" takes place: It is the
branch from which -RELEASE will be "distilled".

The branches freebsd-update can follow are -RELEASE and the
errata branch -RELEASE-pN (where N is the patchlevel). For
following -STABLE or -HEAD, you _have to_ use the source Luke
(except for the snapshots mentioned).



> > > userland first? ok. Got new libs with missing syscals and we can't run
> > > any program.
> > 
> > Dynamic linking to the system's most essential library should
> > not break things. Stable interfaces are very important here,
> > so the upgrader won't be so stupid to shoot his own foot. :-)
> 
> I am talk about reality. Staticaly linked svn, build on 9.x, don't run
> on 6.x system by missinc syscal. This is fact. As result I am build
> svn on 6.x system in VirtualBox.

Of course, because v6 and v9 are not using the same ABI (and API).
This is only guaranteed on -STABLE. If you need to run v6 software
on a v9 system, the compat6x-i386 port, for example, can be used
to privode "downward compatibility"; "upward compatibility" requires
a time machine. :-)



> > Please keep in mind that I'm just mentioning my own thoughts
> > here. I'm not part of the pkg development team. If you have
> > specific questions regarding the use and implementation of
> > the upcoming OS updating mechanism, you should contact the
> > designated maintainers.
> 
> Who maintained this?

Check out "man pkg": "Please direct questions and issues to the
pkg at FreeBSD.org mailing list." as well as its "AUTHORS AND
CONTRIBUTORS" section.

Also see the list of authors here:

https://github.com/freebsd/pkg/blob/master/AUTHORS

They might know (much better than me) how things are going to be
developed, and what plans they have for the future.



-- 
Polytropon
Magdeburg, Germany
Happy FreeBSD user since 4.0
Andra moi ennepe, Mousa, ...