Add custom guest-components build support (#12)

* Add custom guest-components build support

- Add gc_builder stage to Dockerfile.podvm_binaries.ubuntu that compiles
  attestation-agent and api-server-rest from a configurable guest-components
  Git repo/ref when AA_FEATURES or GUEST_COMPONENTS_REF are set.
- Extend Makefile with AA_FEATURES, GUEST_COMPONENTS_REPO, and
  GUEST_COMPONENTS_REF variables to pass custom build parameters.
- Add /run tmpfs mount (80% RAM) to fstab for container image layers.

* Configure kata-agent to read policy from /run/peerpod/policy.rego

Without an explicit policy_file, kata-agent falls back to
/etc/kata-opa/default-policy.rego and ignores the initdata-provided
policy written by process-user-data. The tmpfiles rule already seeds
/run/peerpod/policy.rego with allow-all.rego at boot, and
process-user-data overwrites it when cc_init_data is present.

Made-with: Cursor

* Revert "Configure kata-agent to read policy from /run/peerpod/policy.rego"

This reverts commit 0a4d980.

* Refine mkosi.postinst for systemd unit management

- Corrected the quoting issue in rm commands for legacy mount units to ensure proper deletion.
- Added symlinks to mask unwanted systemd units that could affect vTPM measurements, enhancing security and predictability during VM boot.

This update improves the handling of systemd services related to PCR measurements and ensures a cleaner environment for the container runtime.

* Refactor custom guest-components build to use CUSTOM_GC_BINARIES

Replace hardcoded per-binary build logic with a generic
build-guest-components.sh script driven by a comma-separated
CUSTOM_GC_BINARIES variable. Adding new components (e.g.
confidential-data-hub) now requires only a new case in the
script, with no Dockerfile or Makefile changes.

Made-with: Cursor

* Fix quoting in mkosi.postinst for systemd unit removal

Corrected the escaping of backslashes in rm commands for legacy mount units to ensure proper deletion. This change enhances the reliability of the post-installation script by preventing potential issues with file removal during VM setup.

* Update Dockerfile.mkosi.ubuntu for Ubuntu 24.10 and adjust mkosi.postinst for TDX RTMR support

- Changed base image to Ubuntu 24.10, which includes systemd 256 EFI stub with TDX RTMR support.
- Updated apt sources to redirect to old-releases mirror due to the EOL status of Ubuntu 24.10.
- Clarified comments in mkosi.postinst regarding TDX RTMR support for systemd unit management.

* Enhance build scripts and Dockerfile for guest components

- Updated build-guest-components.sh to include the --locked flag for cargo builds, ensuring consistent dependency resolution.
- Added error handling in Dockerfile.podvm_binaries.ubuntu to require GUEST_COMPONENTS_REF when CUSTOM_GC_BINARIES is specified.
- Clarified Makefile with comments indicating the necessity of GUEST_COMPONENTS_REF when using custom binaries.

* Fix Dockerfile.podvm_binaries.ubuntu to ensure successful installation of binaries by appending 'true' to the loop command, preventing build failures due to non-zero exit codes from the install command.

Author: yousef-cohere

src/cloud-api-adaptor/podvm-mkosi/Dockerfile.mkosi.ubuntu

@@ -1,13 +1,17 @@
 # syntax=docker/dockerfile:1.5.0-labs
-# ubuntu:24.04
-FROM ubuntu@sha256:0d39fcc8335d6d74d5502f6df2d30119ff4790ebbb60b364818d5112d9e3e932 AS builder
+# ubuntu:24.10 (Oracular) — provides systemd 256 EFI stub with TDX RTMR support
+FROM ubuntu@sha256:cdf755952ed117f6126ff4e65810bf93767d4c38f5c7185b50ec1f1078b464cc AS builder
 
 ARG MKOSI_VERSION="v22"
 ARG PROFILE="debug"
 ARG IMAGE_VERSION="0.0.0"
 
 ARG DEBIAN_FRONTEND=noninteractive
 
+# 24.10 is EOL; redirect apt to old-releases mirror
+RUN sed -i 's|archive.ubuntu.com|old-releases.ubuntu.com|g; s|security.ubuntu.com|old-releases.ubuntu.com|g' \
+	/etc/apt/sources.list.d/ubuntu.sources
+
 RUN apt-get update && \
 	apt-get install -y \
 	bubblewrap \

src/cloud-api-adaptor/podvm-mkosi/Makefile

@@ -15,6 +15,23 @@ PODVM_CONTAINER_NAME ?= $(REGISTRY)/podvm-docker-image-$(DISTRO_ARCH)
 VERIFY_PROVENANCE ?= no
 MKOSI_VERSION ?= v22
 
+# Comma-separated list of guest-components binaries to build from source
+# instead of using upstream pre-built OCI artifacts.
+# Recognized: attestation-agent, api-server-rest, confidential-data-hub
+# Example: CUSTOM_GC_BINARIES=attestation-agent,api-server-rest
+CUSTOM_GC_BINARIES ?=
+
+# Cargo features for attestation-agent (required when attestation-agent is
+# listed in CUSTOM_GC_BINARIES).
+# Example: AA_FEATURES=bin,ttrpc,kbs,coco_as,rust-crypto,tdx-attester,nvidia-attester
+AA_FEATURES ?=
+
+# Git repo + ref for guest-components source checkout.
+# GUEST_COMPONENTS_REF is required when CUSTOM_GC_BINARIES is set.
+# Defaults to the OCI reference from versions.yaml via Makefile.defaults.
+GUEST_COMPONENTS_REPO ?= https://github.com/confidential-containers/guest-components.git
+# GUEST_COMPONENTS_REF is inherited from Makefile.defaults (oci.guest-components.reference)
+
 _SHA := $(shell git rev-parse --short HEAD)
 _SHA_DIRTY := $(shell [ -n "$$(git status --porcelain 2>/dev/null)" ] && printf -- '-dirty')
 IMAGE_VERSION ?= $(_SHA)$(_SHA_DIRTY)
@@ -78,6 +95,9 @@ endif
 		--build-arg PAUSE_BIN=$(PAUSE_BIN) \
 		--build-arg IMAGE_NAME=mkosi-podvm-binaries \
 		--build-arg VERIFY_PROVENANCE=$(VERIFY_PROVENANCE) \
+		$(if $(CUSTOM_GC_BINARIES),--build-arg CUSTOM_GC_BINARIES=$(CUSTOM_GC_BINARIES),) \
+		$(if $(AA_FEATURES),--build-arg AA_FEATURES=$(AA_FEATURES),) \
+		$(if $(GUEST_COMPONENTS_REF),--build-arg GUEST_COMPONENTS_REF=$(GUEST_COMPONENTS_REF) --build-arg GUEST_COMPONENTS_REPO=$(GUEST_COMPONENTS_REPO),) \
 		$(if $(AUTHFILE),--build-arg AUTHFILE=$(AUTHFILE),) \
 		$(if $(DEFAULT_AGENT_POLICY_FILE),--build-arg DEFAULT_AGENT_POLICY_FILE=$(DEFAULT_AGENT_POLICY_FILE),) \
 		$(if $(filter $(PUSH),true),,-o type=local,dest="./resources/binaries-tree") \

src/cloud-api-adaptor/podvm-mkosi/mkosi.postinst

@@ -14,8 +14,12 @@ mv "${BUILDROOT}/etc/issue.d" "${BUILDROOT}/usr/lib/issue.d" || true
 } >> "${BUILDROOT}/etc/os-release"
 
 # remove unused units of legacy image
-rm -f "${BUILDROOT}"/etc/systemd/system/{run-image,run-kata\x2dcontainers}.mount
-rm -f "${BUILDROOT}"/etc/systemd/system/multi-user.target.wants/{run-image,run-kata\x2dcontainers}.mount
+# NB: the systemd-escaped filename contains a literal backslash (\x2d = '-').
+# Brace expansion is unquoted, so each alternative must double-escape the
+# backslash to survive bash quote removal.
+rm -f "${BUILDROOT}"/etc/systemd/system/{run-image,run-kata\\x2dcontainers}.mount
+rm -f "${BUILDROOT}"/etc/systemd/system/multi-user.target.wants/{run-image,run-kata\\x2dcontainers}.mount
+
 
 # mask unwanted sytemd units that measure a bunch of stuff into the vTPM
 ln -s /dev/null "${BUILDROOT}/etc/systemd/system/systemd-pcrmachine.service"
@@ -25,3 +29,11 @@ ln -s /dev/null "${BUILDROOT}/etc/systemd/system/systemd-pcrphase@.service"
 ln -s /dev/null "${BUILDROOT}/etc/systemd/system/systemd-pcrphase-initrd.service"
 ln -s /dev/null "${BUILDROOT}/etc/systemd/system/systemd-pcrphase-sysinit.service"
 ln -s /dev/null "${BUILDROOT}/etc/systemd/system/systemd-pcrphase.service"
+
+# Override the target image's systemd-stub with the builder's v256 version.
+# mkosi may use the target's stub for UKI creation; ensure TDX RTMR support is present.
+if [ -f /usr/lib/systemd/boot/efi/linuxx64.efi.stub ]; then
+    mkdir -p "${BUILDROOT}/usr/lib/systemd/boot/efi"
+    cp /usr/lib/systemd/boot/efi/linuxx64.efi.stub \
+       "${BUILDROOT}/usr/lib/systemd/boot/efi/linuxx64.efi.stub"
+fi

src/cloud-api-adaptor/podvm-mkosi/mkosi.skeleton/etc/fstab

@@ -0,0 +1 @@
+tmpfs /run tmpfs rw,nosuid,nodev,size=80% 0 0

src/cloud-api-adaptor/podvm/Dockerfile.podvm_binaries.ubuntu

@@ -5,6 +5,38 @@
 #
 # Build binaries for mkosi podvm image
 #
+# Optional: build guest-components binaries from source.
+# Set CUSTOM_GC_BINARIES to a comma-separated list of components to compile
+# from the guest-components repo instead of using upstream pre-built artifacts.
+# Recognized: attestation-agent, api-server-rest, confidential-data-hub
+#
+# When attestation-agent is included, AA_FEATURES must also be set.
+# GUEST_COMPONENTS_REF controls which git ref to build from.
+FROM rust:1.90-bookworm AS gc_builder
+ARG CUSTOM_GC_BINARIES=""
+ARG AA_FEATURES=""
+ARG GUEST_COMPONENTS_REF=""
+ARG GUEST_COMPONENTS_REPO="https://github.com/confidential-containers/guest-components.git"
+ARG DEBIAN_FRONTEND=noninteractive
+RUN set -e; \
+    if [ -n "${CUSTOM_GC_BINARIES}" ] && [ -z "${GUEST_COMPONENTS_REF}" ]; then \
+      echo "ERROR: GUEST_COMPONENTS_REF must be set when CUSTOM_GC_BINARIES is specified" >&2; \
+      exit 1; \
+    fi; \
+    if [ -n "${CUSTOM_GC_BINARIES}" ] && [ -n "${GUEST_COMPONENTS_REF}" ]; then \
+      apt-get update && \
+      apt-get install -y --no-install-recommends \
+        protobuf-compiler pkg-config clang libssl-dev libtss2-dev && \
+      apt-get clean && rm -rf /var/lib/apt/lists/* && \
+      mkdir -p /build/gc && cd /build/gc && \
+      git init && \
+      git remote add origin "${GUEST_COMPONENTS_REPO}" && \
+      git fetch --depth=1 origin "${GUEST_COMPONENTS_REF}" && \
+      git reset --hard FETCH_HEAD; \
+    fi
+COPY cloud-api-adaptor/podvm/build-guest-components.sh /build/
+RUN /build/build-guest-components.sh "${CUSTOM_GC_BINARIES}" "${AA_FEATURES}"
+
 # ubuntu:24.04
 FROM ubuntu@sha256:0d39fcc8335d6d74d5502f6df2d30119ff4790ebbb60b364818d5112d9e3e932 AS builder
 
@@ -108,5 +140,11 @@ RUN ./hack/cross-build-extras.sh
 
 RUN LIBC=gnu make binaries
 
+COPY --from=gc_builder /output/ /tmp/gc-overrides/
+RUN for bin in /tmp/gc-overrides/*; do \
+      [ -f "$bin" ] && [ -s "$bin" ] && \
+      install -m0755 "$bin" /src/cloud-api-adaptor/podvm/files/usr/local/bin/"$(basename "$bin")"; \
+    done; true
+
 FROM scratch
 COPY --from=podvm_binaries_builder /src/cloud-api-adaptor/podvm/files /

src/cloud-api-adaptor/podvm/build-guest-components.sh

@@ -0,0 +1,54 @@
+#!/bin/bash
+# Build selected guest-components binaries from source.
+#
+# Usage: build-guest-components.sh <comma-separated-binaries> [aa-features]
+#
+# Recognized binaries:
+#   attestation-agent       — requires aa-features argument
+#   api-server-rest
+#   confidential-data-hub
+#
+# Source must already be cloned at /build/gc.
+# Compiled binaries are placed in /output/.
+
+set -euo pipefail
+
+BINARIES="${1:-}"
+AA_FEATURES="${2:-}"
+OUTDIR="/output"
+mkdir -p "$OUTDIR"
+
+[ -z "$BINARIES" ] && exit 0
+
+IFS=',' read -ra BINS <<< "$BINARIES"
+for bin in "${BINS[@]}"; do
+  bin=$(echo "$bin" | xargs)  # trim whitespace
+  case "$bin" in
+    attestation-agent)
+      if [ -z "$AA_FEATURES" ]; then
+        echo "ERROR: attestation-agent requires AA_FEATURES" >&2
+        exit 1
+      fi
+      cd /build/gc/attestation-agent/attestation-agent
+      cargo build --release --locked --no-default-features \
+        --features "$AA_FEATURES" --bin ttrpc-aa
+      cp /build/gc/target/release/ttrpc-aa "$OUTDIR/attestation-agent"
+      ;;
+    api-server-rest)
+      cd /build/gc/api-server-rest
+      cargo build --release --locked
+      cp /build/gc/target/release/api-server-rest "$OUTDIR/api-server-rest"
+      ;;
+    confidential-data-hub)
+      cd /build/gc/confidential-data-hub
+      cargo build --release --locked
+      cp /build/gc/target/release/confidential-data-hub "$OUTDIR/confidential-data-hub"
+      ;;
+    *)
+      echo "ERROR: Unknown guest component: $bin" >&2
+      echo "Recognized: attestation-agent, api-server-rest, confidential-data-hub" >&2
+      exit 1
+      ;;
+  esac
+  echo "Built: $bin"
+done