Problems with packaging /etc and etcupdate(8)
Glen Barber
gjb at FreeBSD.org
Wed Sep 27 16:30:15 UTC 2017
Hi,
Over the past two weeks, I spent quite a bit of time looking into using
etcupdate(8) to bootstrap /var/db/etcupdate for use with pkg(8), and
here are some of the issues I have run into. The most current diff
tested is attached.
1) A new 'create-packages-files' target was added, which invokes the
etcupdate(8) 'extract' command using WSTAGEDIR for the DESTDIR.
Despite passing 'TAGS=package=etc' and 'PACKAGE=etc' to etcupdate(8)
'-M' flag, in addition to '-DNO_ROOT', some files still fail to be
tagged in WSTAGEDIR/var/db/etcupdate/current/METALOG. To work around
this, a hack was added to release/scripts/mtree-to-plist.awk to trap
and tag files when '-v etcupdate=yes' is passed to awk(1) in this
target.
2) Some files do not automatically get created by etcupdate(8), as noted
in the manual page. For example, pwd_mkdb(8), cap_mkdb(1),
newaliases(1), tzsetup(8), and services_mkdb(8) are invoked after
merging files with the newly-created tree. This causes, for example,
files like /etc/passwd to be nonexistent when creating a package. To
work around this, another hack was added to the mtree-to-plist.awk
script to skip nonexistent files in WSTAGEDIR/var/db/etcupdate. This
is still non-ideal, I think.
3) Setting 'prefix = /var/db/etcupdate' in the release/packages/etc.ucl
file appears to be ignored. Files in the 'etc' package conflict with
files installed by other packages, such as runtime, autofs, and so
on, with conflicting files being things like /etc/autofs/include_ldap
or other similarly-installed files within /etc. To work around this
when packaging files, a hack was added to the mtree-to-plist.awk
script (again) to prepend '/var/db/etcupdate' to the file path in
order to get the correct files included in the package. (This is
somewhat in parallel with (2) above.)
4) When running etcupdate(8) after hacking around some of the issues
above, I discovered that /var/db/etcupdate/current in the root of the
system in which it is being run disappears. Additionally, I had to
restore files within /etc (such as master.passwd(5), passwd(5), etc.)
in order to make my test system usable again. In addition to
replacing files that should have been merged, 'pkg check' against the
'FreeBSD-etc' package reports all of the files from the 'current'
directory have vanished.
In addition to the above items, I also did some (albeit minimal) testing
with mergemaster(8), as it creates /var/db/mergemaster.mtree, however in
my brief testing it seems the mtree(8) file created is not compatible
with pkg(8) or the mtree-to-plist.awk script. (Note: the mergemaster(8)
bits from my testing are not included in the diff, as I had to revert
a number of changes to ensure the correct diff was included in this
email.
I am at the point now where I am out of ideas on how to proceed, and
could use some help on figuring this out. Any help on the above issues,
or anything in the Wiki would be greatly appreciated.
https://wiki.freebsd.org/PkgBase
Thanks,
Glen
-------------- next part --------------
Index: Makefile
===================================================================
--- Makefile (revision 324062)
+++ Makefile (working copy)
@@ -132,6 +132,7 @@
builddtb xdev xdev-build xdev-install \
xdev-links native-xtools stageworld stagekernel stage-packages \
create-packages-world create-packages-kernel create-packages \
+ create-files-packages \
packages installconfig real-packages sign-packages package-pkg \
print-dir test-system-compiler
Index: Makefile.inc1
===================================================================
--- Makefile.inc1 (revision 324063)
+++ Makefile.inc1 (working copy)
@@ -1555,6 +1555,7 @@
.ORDER: stage-packages create-packages
.ORDER: create-packages create-world-packages
+.ORDER: create-packages create-files-packages
.ORDER: create-packages create-kernel-packages
.ORDER: create-packages sign-packages
@@ -1596,6 +1597,12 @@
DESTDIR=${WSTAGEDIR} \
PKG_VERSION=${PKG_VERSION} create-world-packages
+create-packages-files: _pkgbootstrap _repodir .PHONY
+ ${_+_}@cd ${.CURDIR}; \
+ ${MAKE} -f Makefile.inc1 \
+ DESTDIR=${WSTAGEDIR} \
+ PKG_VERSION=${PKG_VERSION} create-files-packages
+
create-packages-kernel: _pkgbootstrap _repodir .PHONY
${_+_}@cd ${.CURDIR}; \
${MAKE} -f Makefile.inc1 \
@@ -1603,8 +1610,25 @@
PKG_VERSION=${PKG_VERSION} DISTDIR=kernel \
create-kernel-packages
-create-packages: .PHONY create-packages-world create-packages-kernel
+create-packages: .PHONY create-packages-world create-packages-files create-packages-kernel
+create-files-packages: _pkgbootstrap .PHONY
+ mkdir -p ${WSTAGEDIR}/var/db/etcupdate/current
+ cd ${WSTAGEDIR} ; \
+ ${WSTAGEDIR}/usr/sbin/etcupdate extract -B -M \
+ "TARGET=${TARGET} TARGET_ARCH=${TARGET_ARCH} -DNO_ROOT TAGS=package=etc PACKAGE=etc" \
+ -s ${SRCDIR} -d ${WSTAGEDIR}/var/db/etcupdate
+ awk -f ${SRCDIR}/release/scripts/mtree-to-plist.awk \
+ -v etcupdate=yes -v ws=${WSTAGEDIR} \
+ ${WSTAGEDIR}/var/db/etcupdate/current/METALOG
+ sh ${SRCDIR}/release/packages/generate-ucl.sh -o etc \
+ -s ${SRCDIR} -u ${WSTAGEDIR}/etc.ucl
+ ${PKG_CMD} -o ABI_FILE=${WSTAGEDIR}/bin/sh -o ALLOW_BASE_SHLIBS=yes \
+ create -M ${WSTAGEDIR}/etc.ucl \
+ -p ${WSTAGEDIR}/etc.plist \
+ -r ${WSTAGEDIR} \
+ -o ${REPODIR}/$$(${PKG_CMD} -o ABI_FILE=${WSTAGEDIR}/bin/sh config ABI)/${PKG_VERSION}
+
create-world-packages: _pkgbootstrap .PHONY
@rm -f ${WSTAGEDIR}/*.plist 2>/dev/null || :
@cd ${WSTAGEDIR} ; \
Index: release/packages/etc.ucl
===================================================================
--- release/packages/etc.ucl (nonexistent)
+++ release/packages/etc.ucl (working copy)
@@ -0,0 +1,22 @@
+#
+# $FreeBSD: head/release/packages/runtime.ucl 322327 2017-08-09 19:16:54Z gjb $
+#
+
+name = "FreeBSD-etc"
+origin = "base"
+version = "%VERSION%"
+comment = "etc package"
+categories = [ base ]
+maintainer = "re at FreeBSD.org"
+www = "https://www.FreeBSD.org"
+prefix = "/var/db/etcupdate"
+licenselogic = "single"
+licenses = [ BSD2CLAUSE ]
+desc = <<EOD
+etc package
+EOD
+scripts: {
+ post-install = <<EOD
+ echo "Please use etcupdate(8) or mergemaster(8) to update"
+ echo "files under /etc."
+EOD
Index: release/scripts/mtree-to-plist.awk
===================================================================
--- release/scripts/mtree-to-plist.awk (revision 324062)
+++ release/scripts/mtree-to-plist.awk (working copy)
@@ -28,6 +28,13 @@
tags=tags""_kernconf
}
}
+ if (etcupdate != "") {
+ if (length(gsub(/tags=/, "", tags) == 0))
+ tags="package=etc"
+ if ((length(type) != 0) && (type == "dir")) {
+ next
+ }
+ }
if (length(tags) == 0)
next
if (tags ~ /package=/) {
@@ -76,5 +83,17 @@
output=pkgname".plist"
}
+ if (etcupdate != "") {
+ pfx="/var/db/etcupdate/current"
+ filename=ws""pfx""$1
+ if (system("test -e " filename) == 0) {
+ print "@"type"("uname","gname","mode","flags") " pfx""$1 > output
+ next
+ } else {
+ next
+ }
+ next
+ }
+
print "@"type"("uname","gname","mode","flags") " $1 > output
}
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.freebsd.org/pipermail/freebsd-pkgbase/attachments/20170927/e12995c0/attachment.sig>
More information about the freebsd-pkgbase
mailing list