GSoC: Separation of Ports Build Process from Local Installation

[Theron Tarigo]

Hello all,

This project aims to decouple the build process of the ports framework 
from the local installation.  To work around the assumptions made by 
ports about dependency file locations, a userspace tool to remap 
processes' filesystem namespaces is under development: 
freebsd-user-namespace, provided as port devel/userns.

This past week, I have made the following progress:
- Modified bsd.port.mk to run pre-* do-* post-* ports targets in a 
sub-make in userns environment, only when needed (when a port defines 
those targets itself).
- Fixed some non-fatal errors typically seen during dependency installation.
- Added support to userns's execve implementation to run ELF executables 
which use DT_RUNPATH by translating this to an appropriate LD_LIBMAP 
environment value.
- Attributed some failures to a faulty FreeBSD installation on my test 
system, which is now fixed.

Latest summary of port building success:
- 5610 ports considered for testing (2000 requested + their dependencies)
- 4321 (77%) succeeded
- 140 (2%) failed
- 1149 (20%) skipped due to failed dependencies
- Success rate: 97% (portion succeeded of those which were started, not 
skipped)

Compare to last summary:
- 5613 ports tried
- 2958 (53%) succeeded
- 177 (3%) failed
- 2478 (44%) skipped due to failed dependencies
- Success rate: 94%

Although success rate is similar, overall success is much higher (77% 
vs. 53%) due to significantly fewer failed dependencies.

What I am working on next:

- Fix known issues in userns: resolve symbolic link targets, resolve 
".." in pathnames.
- Eliminate remaining dependency failures.
- Compare the packages built by separated-build method to either 
official packages or ports built by original FreeBSD method.  Any 
differences should be attributable to built times and hostnames.
- Provide more thorough documentation of PORTS_SEPARATED_BUILD, in 
particular its usage and its limitations, so that users and devs 
interested in this project can more easily test it for themselves.

Project goals and status are kept at 
https://wiki.freebsd.org/SummerOfCode2019Projects/PortsSeparatedBuild

Source of the userspace filesystem namespace tool is shared at 
https://github.com/therontarigo/freebsd-user-namespace .
Changes to ports framework are shared at 
https://github.com/freebsd/freebsd-ports/compare/master...therontarigo:separated.

Theron Tarigo