From nobody Tue Apr 22 20:35:23 2025 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4ZhvBM36FYz5t9Vd; Tue, 22 Apr 2025 20:35:23 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4ZhvBM2Swjz3Yk3; Tue, 22 Apr 2025 20:35:23 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1745354123; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=iMXZCGIjYepneImw4Vuxz/cED9/mfkZwM9ecmSvAND0=; b=mKN2KwNSjuk9HP1IJdqJVjqtiNrYSANDM1Hvhe2Sv/NvTWXZBt7xgS+83bmaRbHzmflcZw M4q1Z8x3GI0Y0JLxJiHCxGOXZsObOhdT1iiT1YUCfZVv+3SYyzxGDv+HPKP7MgDm3liSD7 KOML+uiRCCO2QffPFQCahxZP5qxC2ei15Y+KE2G0izxH+5g1LfOm66n6uqgcOeFH8WRD78 0+KBMxh3Dmt+Pvjl/osqE5rBqsCJ93T11JHJEeuD/hCGo81OGcdR5lUwJwPm6r+JW0glys RaWSX6swFxK5zz069bQoR9f2jE5zmfXqBy91Ys2cnkrIiL82zOldF51C5Mo97A== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1745354123; a=rsa-sha256; cv=none; b=juCqm907NszTUmZE9f5InRCBhMveUK/V/K0w/tzaSZS7gTug+XOud1yv6iuzcANm5DEOsx wEVcQmC0CMRk+gq4CjRBzIpnJMwMblZINJx3TxlaNLIDO05ISz8tN2gyImVgno5uzRRrCV WjyzG7kNGGT3Zaqzag+goa1rAd1Zgj0TfoUasgOE7csPAXZ/NefTmJXF4DXGYJmV/nhS98 pwrIDBXu9fTJ1rh2CAs9ZWmvvHkg2oQ4RfVhwkZSdpr4O0GHaZ1g0mVkMxydT+VL2irXnm T4dZpMBC7YoZ2YNjWuIuiFuslv0BC7Cmtf82p7F51z5Fe93gCxlONPg/HyxjXg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1745354123; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=iMXZCGIjYepneImw4Vuxz/cED9/mfkZwM9ecmSvAND0=; b=bfAC07gQE/maLpw20G4rZK5hCA2P+oGTaoDU9fHvhC3UBIERq+Nw3/K4zyo83531C4edAE Jl6Ps969baS0OiGujnGhhX2f+NDLLxRlxnELV9+ZYbNHt/16sv7u8jDQazh0h1lP9CpIh6 HfKie2/aMlB7fLqvJafhyS0UP2PI6h4N8BNbZLh35CM9Stf9G8XuefghLyQhZRo637wBf5 +G/KoBRPN0Lpz/hfFcVK8qRUZjnhCvdBEVJ89gFMRSLX+FdIf0GDxxdcynwVDWiunH47T3 AXJG5mkmVipSswDJjnkhq6AdoRFrwJ5vTkw8FaCU3I0igkW1vYlK4q8UeqQEMQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4ZhvBM1ZlBzZWr; Tue, 22 Apr 2025 20:35:23 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 53MKZNf5055347; Tue, 22 Apr 2025 20:35:23 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 53MKZNVU055344; Tue, 22 Apr 2025 20:35:23 GMT (envelope-from git) Date: Tue, 22 Apr 2025 20:35:23 GMT Message-Id: <202504222035.53MKZNVU055344@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Colin Percival Subject: git: 584265890303 - main - EC2: Add AMI Builder AMI building List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: cperciva X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 58426589030308cd632477d328b9536b1634c54d Auto-Submitted: auto-generated The branch main has been updated by cperciva: URL: https://cgit.FreeBSD.org/src/commit/?id=58426589030308cd632477d328b9536b1634c54d commit 58426589030308cd632477d328b9536b1634c54d Author: Colin Percival AuthorDate: 2025-04-20 16:38:48 +0000 Commit: Colin Percival CommitDate: 2025-04-22 20:35:03 +0000 EC2: Add AMI Builder AMI building Starting in 2015 I have published "AMI Builder AMIs" for FreeBSD/EC2: These boot into a memory disk, extract a "clean" copy of FreeBSD onto the root disk, mount it at /mnt, and allow the user to SSH in to make customizations before creating a new AMI from the "running" instance (in fact, from the FreeBSD installation which is not running but is mounted on /mnt). This provides a much cleaner mechanism for building customized FreeBSD AMIs than the traditional Linux approach of "launch an EC2 instance, SSH in and configure it, then try to wipe logs and credentials before creating an AMI"; and it's easier than building a customized AMI ab initio by modifying the FreeBSD release-building code. This commit brings that functionality into the FreeBSD src tree and into the collection of images built by the release engineering team: The EC2 "BUILDER" flavour AMI is essentially a "SMALL" flavour AMI with a compressed "BASE" flavour disk image, plus an init script which juggles disks around (rerooting into a memory disk and extracting the "BASE" image onto disk). Polished by: bz, emaste MFC after: 1 week Sponsored by: Amazon Differential Revision: https://reviews.freebsd.org/D49930 --- release/Makefile.vm | 13 ++++++-- release/tools/ec2-builder.conf | 57 ++++++++++++++++++++++++++++++++ release/tools/mkami.sh | 65 +++++++++++++++++++++++++++++++++++++ release/tools/rc.amibuilder | 74 ++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 207 insertions(+), 2 deletions(-) diff --git a/release/Makefile.vm b/release/Makefile.vm index 0b5f5e714cc5..7821109cf9ca 100644 --- a/release/Makefile.vm +++ b/release/Makefile.vm @@ -42,8 +42,9 @@ BASIC-CLOUDINIT_FSLIST?= ufs zfs BASIC-CLOUDINIT_DESC?= Images for VM with cloudinit disk config support EC2_FORMAT= raw EC2_FSLIST?= ufs zfs -EC2_FLAVOURS?= BASE CLOUD-INIT SMALL +EC2_FLAVOURS?= BASE BUILDER CLOUD-INIT SMALL EC2-BASE_DESC= Amazon EC2 image +EC2-BUILDER_DESC= Amazon EC2 AMI Builder image EC2-CLOUD-INIT_DESC= Amazon EC2 Cloud-Init image EC2-SMALL_DESC= Amazon EC2 small image GCE_FORMAT= raw @@ -124,14 +125,22 @@ CLEANFILES+= ${_CW:tl}.${_FS}.${_FMT}.img \ ${_CW:tl}.${_FS}.${_FMT}.raw ${_CW:tu}${_FS:tu}${_FMT:tu}IMAGE= ${_CW:tl}.${_FS}.${_FMT} +# Special handling: EC2 "AMI Builder" images need a copy of the "base" disk +# built first. +.if ${_CW} == EC2-BUILDER +cw-${_CW:tl}-${_FS}-${_FMT}: cw-ec2-base-${_FS}-${_FMT} +.endif + cw-${_CW:tl}-${_FS}-${_FMT}: ${QEMUTGT} mkdir -p ${.OBJDIR}/${.TARGET} env TARGET=${TARGET} TARGET_ARCH=${TARGET_ARCH} SWAPSIZE=${SWAPSIZE} \ QEMUSTATIC=${QEMUSTATIC} \ + EC2BASEIMG=${.OBJDIR}/${EC2-BASE${_FS:tu}${_FMT:tu}IMAGE} \ ${WITHOUT_QEMU:DWITHOUT_QEMU=true} \ ${NO_ROOT:DNO_ROOT=true} \ ${.CURDIR}/scripts/mk-vmimage.sh \ - -C ${.CURDIR}/tools/vmimage.subr -d ${.OBJDIR}/${.TARGET} -F ${_FS} \ + -C ${.CURDIR}/tools/vmimage.subr -d ${.OBJDIR}/${.TARGET} \ + -F ${"${_CW:MEC2-BUILDER}" != "":?ufs:${_FS}} \ -i ${.OBJDIR}/${_CW:tl}.${_FS}.${_FMT}.img -s ${VMSIZE} -f ${_FMT} \ -S ${WORLDDIR} -o ${.OBJDIR}/${${_CW:tu}${_FS:tu}${_FMT:tu}IMAGE} -c ${${_CW:tu}CONF} touch ${.TARGET} diff --git a/release/tools/ec2-builder.conf b/release/tools/ec2-builder.conf new file mode 100644 index 000000000000..7927fc13a74d --- /dev/null +++ b/release/tools/ec2-builder.conf @@ -0,0 +1,57 @@ +#!/bin/sh + +. ${WORLDDIR}/release/tools/ec2.conf + +# Build with a 7.9 GB partition; this is enough for our stripped-down +# base system plus the compressed ec2-base image. +export VMSIZE=8000m + +# Flags to installworld/kernel: We don't want debug symbols (kernel or +# userland), 32-bit libraries, tests, or the debugger. +export INSTALLOPTS="WITHOUT_DEBUG_FILES=YES WITHOUT_KERNEL_SYMBOLS=YES \ + WITHOUT_LIB32=YES WITHOUT_TESTS=YES WITHOUT_LLDB=YES" + +# Packages to install into the image we're creating. In addition to packages +# present on all EC2 AMIs, we install: +# * ec2-scripts, which provides a range of EC2ification startup scripts, +# * isc-dhcp44-client, used for IPv6 network setup, and +# * py-awscli, to make it easier for users to create AMIs. +export VM_EXTRA_PACKAGES="${VM_EXTRA_PACKAGES} ec2-scripts \ + isc-dhcp44-client devel/py-awscli" + +# Services to enable in rc.conf(5). +export VM_RC_LIST="${VM_RC_LIST} ec2_configinit ec2_ephemeral_swap \ + ec2_fetchkey ec2_loghostkey sshd" + +vm_extra_pre_umount() { + # Any EC2 ephemeral disks seen when the system first boots will + # be "new" disks; there is no "previous boot" when they might have + # been seen and used already. + touch ${DESTDIR}/var/db/ec2_ephemeral_diskseen + + # Configuration common to all EC2 AMIs + ec2_common + + # Standard FreeBSD network configuration + ec2_base_networking + + # Grab a copy of the ec2-base disk image, and compress it + zstd < ${EC2BASEIMG} > ${DESTDIR}/image.zst + + # Disable fortune so we don't have extra noise at login + chmod a-x ${DESTDIR}/usr/bin/fortune + + # Install the AMI-building script + install -m 755 ${WORLDDIR}/release/tools/mkami.sh ${DESTDIR}/bin/mkami + + # Install an /etc/rc which juggles disks around for us + install -m 755 ${WORLDDIR}/release/tools/rc.amibuilder ${DESTDIR}/etc + + # We want to mount from the UFS disk and juggle disks first + cat >> ${DESTDIR}/boot/loader.conf <<-EOF + vfs.root.mountfrom="ufs:/dev/gpt/rootfs" + init_script="/etc/rc.amibuilder" + EOF + + return 0 +} diff --git a/release/tools/mkami.sh b/release/tools/mkami.sh new file mode 100644 index 000000000000..cfbbcd3bc8a9 --- /dev/null +++ b/release/tools/mkami.sh @@ -0,0 +1,65 @@ +#!/bin/sh -e +# +# Copyright (c) 2015 Colin Percival +# +# SPDX-License-Identifier: BSD-2-Clause +# +# mkami.sh: Create an AMI from the currently running EC2 instance. +# + +export PATH=$PATH:/usr/local/bin + +NAME=$1 +if [ -z "$NAME" ]; then + echo "usage: mkami []" + exit 1 +fi +DESC=$2 +if ! [ -z "$DESC" ]; then + DESCOPT="--description '$DESC'" +fi + +# Get the instance ID and region from the EC2 Instance Metadata Service: +# https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html +TMPFILE=`mktemp` +fetch -qo $TMPFILE http://169.254.169.254/latest/dynamic/instance-identity/document +INST=`awk -F \" '/"instanceId"/ { print $4 }' $TMPFILE` +REGION=`awk -F \" '/"region"/ { print $4 }' $TMPFILE` +rm $TMPFILE +CMD="aws --region $REGION ec2 create-image --instance-id $INST --output text --no-reboot --name '$NAME' $DESCOPT" + +# Unmount the new system image +if mount -p | grep -q '/mnt.*ufs'; then + echo -n "Unmounting new system image..." + sync + umount /mnt + sync + sleep 5 + sync + echo " done." +elif mount -p | grep -q '/mnt.*zfs'; then + echo -n "Unmounting new system image..." + sync + zfs umount -a + zfs umount zroot/ROOT/default + sync + sleep 5 + sync + echo " done." +fi + +if eval "$CMD" --dry-run 2>&1 | + grep -qE 'UnauthorizedOperation|Unable to locate credentials'; then + echo "This EC2 instance does not have permission to create AMIs." + echo "Launch an AMI-builder instance with an appropriate IAM Role," + echo "create an AMI from this instance via the AWS Console, or run" + echo "the following command from a system with the necessary keys:" + echo + echo "$CMD" + exit +fi + +echo -n "Creating AMI..." +AMINAME=`eval "$CMD"` +echo " done." +echo "AMI created in $REGION: $AMINAME" diff --git a/release/tools/rc.amibuilder b/release/tools/rc.amibuilder new file mode 100644 index 000000000000..b25e8facb597 --- /dev/null +++ b/release/tools/rc.amibuilder @@ -0,0 +1,74 @@ +#!/bin/sh +# +# Copyright (c) 2025 Colin Percival +# +# SPDX-License-Identifier: BSD-2-Clause +# +# rc.amibuilder: Juggle disks to reroot into a memory disk and install a clean +# copy of FreeBSD onto the root disk. +# + +# Do nothing if init(8) is in the middle of rerooting +if ps -p 1 -o command | grep -q reroot; then + exit 0 +fi + +# Figure out which partition we boot from +BOOTPART=$(sysctl -n kern.geom.conftxt | + grep -E 'PART|gpt/rootfs' | + grep -B 1 gpt/rootfs | + awk '{ print $3 }' | + head -1) +BOOTDISK=${BOOTPART%%p*} +BOOTPARTNUM=${BOOTPART##*p} + +# First pass: Copy ourselves into a memory disk and reroot into it +if ! [ -c /dev/md0 ]; then + # Create a memory disk of appropriate size and copy the disk + echo "Copying FreeBSD into memory disk..." + DISKBYTES=$(diskinfo ${BOOTDISK} | awk '{print $3}') + mdconfig -a -t swap -s ${DISKBYTES}b -u 0 + dd if=/dev/${BOOTDISK} of=/dev/md0 bs=1M + + # Reboot into the memory disk we just created + echo "Rebooting into memory disk..." + kenv vfs.root.mountfrom="ufs:/dev/md0p${BOOTPARTNUM}" + reboot -r + + # Lose a race against init + sleep 10 + exit 1 +fi + +# Second pass: Extract a clean copy of FreeBSD onto the physical disk +echo "Installing base FreeBSD image..." +sysctl kern.geom.debugflags=16 +zstdcat < /image.zst | dd bs=1M of=/dev/${BOOTDISK} + +# Mount the clean image +if gpart show ${BOOTDISK} | grep -q freebsd-ufs; then + mount /dev/${BOOTPART} /mnt +else + zpool import -aNR /mnt + zfs mount zroot/ROOT/default + zfs mount -a +fi + +# Provide instructions for when the user logs in +mount -w / +cat >/etc/motd.template < [] + +to create the AMI. Don't forget to shut down this instance when +you're done! + +EOF +mount -o ro / + +# After we exit, the boot proceeds with init spawning /etc/rc normally +exit 0