GSoC: Separation of Ports Build Process from Local Installation

[Bakul Shah]

Is there any way Theron can have access to a set of build machines? That would allow him to do some regression testing as well as try out his scheme on a much larger number of ports.

> On Aug 6, 2019, at 11:01 PM, Theron Tarigo <theron at freebsd.org> wrote:
> 
> Hello all,
> 
> This project aims to decouple the build process of the ports framework from the local installation.  To work around the assumptions made by ports about dependency file locations, a userspace tool to remap processes' filesystem namespaces is under development: freebsd-user-namespace, provided as port devel/userns.
> 
> This past week, I have made the following progress:
> - Modified bsd.port.mk to run pre-* do-* post-* ports targets in a sub-make in userns environment, only when needed (when a port defines those targets itself).
> - Fixed some non-fatal errors typically seen during dependency installation.
> - Added support to userns's execve implementation to run ELF executables which use DT_RUNPATH by translating this to an appropriate LD_LIBMAP environment value.
> - Attributed some failures to a faulty FreeBSD installation on my test system, which is now fixed.
> 
> Latest summary of port building success:
> - 5610 ports considered for testing (2000 requested + their dependencies)
> - 4321 (77%) succeeded
> - 140 (2%) failed
> - 1149 (20%) skipped due to failed dependencies
> - Success rate: 97% (portion succeeded of those which were started, not skipped)
> 
> Compare to last summary:
> - 5613 ports tried
> - 2958 (53%) succeeded
> - 177 (3%) failed
> - 2478 (44%) skipped due to failed dependencies
> - Success rate: 94%
> 
> Although success rate is similar, overall success is much higher (77% vs. 53%) due to significantly fewer failed dependencies.
> 
> What I am working on next:
> 
> - Fix known issues in userns: resolve symbolic link targets, resolve ".." in pathnames.
> - Eliminate remaining dependency failures.
> - Compare the packages built by separated-build method to either official packages or ports built by original FreeBSD method.  Any differences should be attributable to built times and hostnames.
> - Provide more thorough documentation of PORTS_SEPARATED_BUILD, in particular its usage and its limitations, so that users and devs interested in this project can more easily test it for themselves.
> 
> Project goals and status are kept at https://wiki.freebsd.org/SummerOfCode2019Projects/PortsSeparatedBuild <https://wiki.freebsd.org/SummerOfCode2019Projects/PortsSeparatedBuild> 
> 
> Source of the userspace filesystem namespace tool is shared at https://github.com/therontarigo/freebsd-user-namespace <https://github.com/therontarigo/freebsd-user-namespace> . 
> Changes to ports framework are shared at https://github.com/freebsd/freebsd-ports/compare/master...therontarigo:separated <https://github.com/freebsd/freebsd-ports/compare/master...therontarigo:separated>.
> 
> Theron Tarigo
>