Creating jails from pkgbase packages
Martin Jakob
mj-mailinglist at gmx.de
Tue Apr 30 11:24:16 UTC 2019
My first attempt seems to be empty when its arrived at the maillinglist, so i try it again in pure text format (i hope). Sorry, if there are unwanted duplicates.
Since a few days i was experimenting with jails build from pkgbase (i got the idea from Michael W. Lucas Freebsd Jails book). Now there is some discussion about pkgbase again :)
So i want to share my experience, i hope this is not offtopic to this conversation and my writing style is not too offputting...
This experiments were made on an resently updated 12-stable system. Sorry, i dont have a system with head available right now.
Anyway, this is what i did:
- I first set the variable "REPODIR=/usr/repo" in /etc/make.conf, to have a nonvolatile repository (is this the right place/variable to do this?).
- then after the standard "make buildworld|installworld|buildkernel|installkernel" in /usr/src, i created the pkgbase packages with the command "make packages".
the files are placed in "/usr/repo/FreeBSD:12:amd64/latest" where latest is a symlink to the, well latest build.
drwxr-xr-x 2 root wheel 805 14 Apr. 14:59 12.0.s20190414123806/
drwxr-xr-x 2 root wheel 805 21 Apr. 09:27 12.0.s20190419155807/
drwxr-xr-x 2 root wheel 805 22 Apr. 13:53 12.0.s20190422094219/
drwxr-xr-x 2 root wheel 805 27 Apr. 14:12 12.0.s20190427084356/
lrwxr-xr-x 1 root wheel 20 27 Apr. 11:04 latest@ -> 12.0.s20190427084356
- this repo is published via the file "/usr/local/etc/pkg/repos/FreeBSD-base.conf"which has this content:
root at betablock:~ # cat "/usr/local/etc/pkg/repos/FreeBSD-base.conf"
# FreeBSD base system repository
FreeBSD-base: {
url: "file:///usr/repo/${ABI}/latest",
mirror_type: "none",
enabled: yes
}
now to the jail stuff:
- this command installs the FreeBSD-runtime and the 72 neccesary dependencies in the jails root directory /jails/test03:
root at betablock:~ # pkg --rootdir /jails/test03 -o 'ASSUME_ALWAYS_YES=true' install -r FreeBSD-base FreeBSD-runtime
- the newly created jail "test03" (an entry in /etc/jail.conf already exists) is started with the command:
root at betablock:~ # jail -vc test03
sadly the jail wont start, it exits with this message:
...
test03: created
test03: run command in jail: /bin/sh /etc/rc
jail: test03: getpwnam: No such file or directory
jail: test03: /bin/sh /etc/rc: failed
test03: removed
...
This happens, because there are no user-database files (like master.passwd, pwd.db, spwd.db) and no group files in the jails /etc/ directory. hmmm...
I saw in the pkg install output of the FreeBSD-runtime pkg this message:
...
[2/72] Extracting FreeBSD-runtime-12.0.s20190414123806: 100%
pwd_mkdb: /jails/test03/etc/master.passwd: No such file or directory
...
so pkg tries to run the pwd_mkdb as defined in the runtime.ucl file, or embedded in the pkg file, it is viewable with this command:
root at betablock:~ # pkg info --raw --file /usr/repo/FreeBSD:12:amd64/latest/FreeBSD-runtime-12.0.s20190414123806.txz
...
scripts: {
post-install: "cap_mkdb -l ${PKG_ROOTDIR}/etc/login.conf\n\tpwd_mkdb -i -p -d ${PKG_ROOTDIR}/etc ${PKG_ROOTDIR}/etc/master.passwd\n\tservices_mkdb -l -q -o ${PKG_ROOTDIR}/var/db/services.db ${PKG_ROOTDIR}/etc/services\n\tchmod 1777 ${PKG_ROOTDIR}/tmp"
}
...
- so for a quick fix, i copied the "missing" files (master.passwd, pwd.db, spwd.db, group) from a FreeBSD base.txz package to my jails etc directory and tried to start the jail again. Read below for a "better" solution.
Now the jail starts:
root at betablock:~ # jls
JID IP Address Hostname Path
...
5 192.168.0.113 test03.local /jails/test03
i can "enter" it with this command:
root at betablock:~ # jexec -l test03
now to some more experiments with the jail which results in some questions (also, see below)
- trying to ping something:
root at test03:~ # ping google.com
ld-elf.so.1: Shared object "libcap_dns.so.1" not found, required by "ping"
the required library libcap_dns.so.1 is in the file FreeBSD-libcasper-12.0.s20190414123806.txz, so lets install it:
Outside of the jail:
root at betablock:~ # pkg --rootdir /jails/test03 -o 'ASSUME_ALWAYS_YES=true' install -r FreeBSD-base FreeBSD-libcasper
and inside the jail:
root at test03:~ # ping google.com
PING google.com (216.58.213.206): 56 data bytes
64 bytes from 216.58.213.206: icmp_seq=0 ttl=52 time=23.398 ms
64 bytes from 216.58.213.206: icmp_seq=1 ttl=52 time=23.975 ms
64 bytes from 216.58.213.206: icmp_seq=2 ttl=52 time=23.252 ms
...
success!
The size of this freshly installed jail:
du -h -d 1 /jails/
...
90M /jails/test03
- now lets install some packages from outside of the jail:
root at betablock:~ # pkg --rootdir /jails/test03 install nginx-full
Updating FreeBSD repository catalogue...
pkg: Repository FreeBSD load error: access repo file(/jails/test03/var/db/pkg/repo-FreeBSD.sqlite) failed: No such file or directory
Fetching meta.txz: 100% 944 B 0.9kB/s 00:01
Fetching packagesite.txz: 100% 6 MiB 2.2MB/s 00:03
Processing entries: 100%
FreeBSD repository update completed. 31883 packages processed.
Updating Synth repository catalogue...
Synth repository is up to date.
Updating FreeBSD-base repository catalogue...
FreeBSD-base repository is up to date.
All repositories are up to date.
The following 146 package(s) will be affected (of 0 checked):
New packages to be INSTALLED:
nginx-full: 1.16.0_2,2 [FreeBSD]
apache24: 2.4.39 [FreeBSD]
libnghttp2: 1.38.0 [FreeBSD]
...
libdrizzle: 0.8_6 [FreeBSD]
brotli: 1.0.7_1,1 [FreeBSD]
Number of packages to be installed: 146
The process will require 953 MiB more space.
232 MiB to be downloaded.
...
Funfact - the jailsize after this installation:
1,0G /jails/test03
- lets enable the nginx service
root at betablock:~ # sysrc -R /jails/test03 nginx_enable=YES
nginx_enable: -> YES
- and start it (after a jail restart, without it nginx does not find its libraries, installed above)
root at test03:~ # service nginx start
Performing sanity check on nginx configuration:
ld-elf.so.1: Shared object "libexecinfo.so.1" not found, required by "libprofiler.so.0"
- lets install this missing library with the command
root at betablock:~ # pkg --rootdir /jails/test03 -o 'ASSUME_ALWAYS_YES=true' install -r FreeBSD-base FreeBSD-libexecinfo
- and lets also install vi:
root at betablock:~ # pkg --rootdir /jails/test03 -o 'ASSUME_ALWAYS_YES=true' install -r FreeBSD-base FreeBSD-vi
- so lets try again (after creating the missing config files nginx.conf and mime.types):
root at betablock:~ # jexec -l test03 service nginx start
Starting nginx.
nginx: [emerg] gethostbyname() failed in /usr/local/etc/nginx/nginx.conf:122
- and the check:
root at betablock:~ # jexec -l test03 service nginx status
nginx is running as pid 41359.
(Browser access works too...)
- Now Lets work with packages within the jail:
root at test03:~ # pkg update
The package management tool is not yet installed on your system.
Do you want to fetch and install it now? [y/N]: y
Bootstrapping pkg from pkg+http://pkg.FreeBSD.org/FreeBSD:12:amd64/latest, please wait...
pkg: Error loading revoked certificates
- Hm, this is because the directory "/usr/share/keys/pkg/revoked/" is missing, lets create it:
root at test03:~ # mkdir /usr/share/keys/pkg/revoked/
- and now:
root at test03:~ # pkg update
The package management tool is not yet installed on your system.
Do you want to fetch and install it now? [y/N]: y
Bootstrapping pkg from pkg+http://pkg.FreeBSD.org/FreeBSD:12:amd64/latest[http://pkg.FreeBSD.org/FreeBSD:12:amd64/latest], please wait...
Verifying signature with trusted certificate pkg.freebsd.org.2013102301... done
[test03.local] Installing pkg-1.10.5_5...
[test03.local] Extracting pkg-1.10.5_5: 100%
Updating FreeBSD repository catalogue...
FreeBSD repository is up to date.
All repositories are up to date.
root at test03:~ # pkg info
FreeBSD-casper-12.0.s20190414123806 casper package
FreeBSD-clibs-12.0.s20190414123806 Core C Libraries
FreeBSD-jail-12.0.s20190414123806 Jail Utilities
FreeBSD-lib-12.0.s20190414123806 lib package
FreeBSD-lib80211-12.0.s20190414123806 lib80211 package
...
x265-3.0 H.265/High Efficiency Video Coding (HEVC) format
xorg-fonts-truetype-7.7_1 X.Org TrueType fonts
xorgproto-2018.4 xorg protocol headers
xvid-1.3.5,1 Opensource MPEG-4 codec, based on OpenDivx
yajl-2.1.0 Portable JSON parsing and serialization library in ANSI C
Btw. The missing "revoked" directory problem shoud be fixed with https://reviews.freebsd.org/D20056[https://reviews.freebsd.org/D20056]
---
So this is a basic jail installation with pkgbase, now to some further experiments and resulting questions:
- i looked into why the userdatabase was missing in the newly created jail. Well, it is, because the files are not included in the default Freebsd-runtime package.
To check if it would work, i built my own Freebsd-runtime package with these files added. to do this, i copied the runtime.plist file from the $WSTAGEDIR and modified it.
- first i removed a lot of duplicated directory entries like
@dir(root,wheel,0755,) /boot
@dir(root,wheel,0755,) /boot
@dir(root,wheel,0755,) /boot
@dir(root,wheel,0755,) /boot
... (and many more)
- then i added the "missing" files and directory
@config(root,wheel,0600,) /etc/master.passwd
@config(root,wheel,0644,) /etc/group
@dir(root,wheel,0755,) /usr/share/keys/pkg/revoked
- after that i rebuild the Freebsd-runtime package with these changes (found in /usr/src/Makefile.inc1):
root at betablock:~ # pkg -o ABI_FILE=/usr/obj/usr/src/amd64.amd64/worldstage/usr/bin/uname -o ALLOW_BASE_SHLIBS=yes create -M /usr/obj/usr/src/amd64.amd64/worldstage/runtime.ucl -p /root/pkgs/runtime.plist -r /usr/obj/usr/src/amd64.amd64/worldstage --output-dir /usr/repo/FreeBSD:12:amd64/latest
and rebuilding the repo metadata with:
root at betablock:~ # pkg repo /usr/repo/FreeBSD:12:amd64/latest
a jail created with this Freebsd-runtime starts directly, without the steps mentioned above.
My Question: Is the ommission of these files intentional?
- one last experiment: i deleted the manpages, the examples, and the /boot directory (i guessed a jail has no need for it), the jail build from this file is 75M, the installation of packages like nginx works like in the original file.
- should the FreeBSD-libcasper package be installed as a dependency of the FreeBSD-casper package?
- i did not observe more missing libraries like libcasper above, but depending on the tasks that are executed or packages that are installed, there might be more that are missing.
So that was my pkgbase-jail experiment, it was fun and i learned some things about pkgbase. Any comments, tips, tricks and (mild) critique are welcome.
Greetings
Martin
More information about the freebsd-pkgbase
mailing list