The Air-Gap Constraint That Fixes Connected Deployments

▶ Watch (04:50)

Shipping an application to an air-gapped environment forces you to include everything: application, dependencies, images. Nothing gets left behind or else something breaks. Brandt Keller argued that constraint solves the portability problem all the way back to connected environments. If a single Zarf package deploys deterministically to a disconnected network, it deploys deterministically anywhere. One package, any environment, with no per-environment Helm value overrides. The developer encodes deployment knowledge at package-create time. The operator runs zarf package deploy and gets a running application.

Declaring Every Dependency in zarf.yaml

▶ Watch (15:01)

The zarf.yaml is the manifest that drives everything. It specifies a kind: ZarfPackageConfig, a metadata block with name and version, and components. Each component lists Helm chart sources (OCI registry or index URL), associated values files, and images. For GitOps deployments using Flux or Argo CD, it also lists git repositories. zarf find-images templates the Helm chart and extracts image references automatically, outputting a ready-to-paste images array. Helm rendering runs at deploy time so operators can pass variables that adjust configuration per environment.

Package Create: One tar.zst With Everything Inside

▶ Watch (19:55)

zarf package create . reads the zarf.yaml, pulls every Helm chart, OCI image, and git repository, generates a Syft SBOM for each, checksums every file, and writes everything into a single tar.zst archive. The archive name includes package name, version, and architecture. During the tutorial, Docker Hub rate limits hit the full room simultaneously. That failure illustrated a benefit: once deployed, images sit in the cluster’s internal registry. A DockerHub or GitHub outage doesn’t affect pods that already made it through the first deploy.

Deploy and Mutate: No Configuration Changes Required

▶ Watch (38:20)

zarf package deploy pushes images from the tar.zst into the cluster’s internal distribution registry, clones bundled git repos into an in-cluster Gitea server, then runs the Helm install. A mutating webhook intercepts pod specs and Argo/Flux resources, rewrites image references to point at the internal registry, and rewrites git repo URLs to point at Gitea. No Helm values change. No per-environment configuration. The deployer runs one command and the result is a running Argo CD with all its images already present locally.

Checksums, Signing, and SBOMs: Integrity from the Start

▶ Watch (45:41)

Every file in a Zarf package gets an individual checksum. Those checksums go into checksums.txt. The zarf.yaml inside the package carries an aggregate checksum of checksums.txt, forming a Merkle tree. A mismatch at deploy time triggers immediate failure. Package signing goes further: signing the zarf.yaml chains the signature to the aggregate checksum and therefore to every bundled artifact. The --enforce flag makes unsigned packages undeployable. zarf package inspect sbom extracts Syft-generated SBOMs as JSON or browsable HTML, visible before any deployment decision.

Q&A

Can you replace the Zarf-managed registry with an external one? Yes. Pass your registry URL and credentials to zarf init and the mutation webhook points image refs at your registry instead. ▶ 24:00

Does Zarf support Kustomize? Yes, use the manifests key instead of charts in a component; Kustomize templating runs at create time, not deploy time. ▶ 27:51

Are multi-architecture images supported? Not natively. Zarf blocks index SHA to prevent pulling every arch variant; the team has an open issue for an opt-in flag. ▶ 59:02

Key Takeaways

  • Solving air-gap packaging also solves connected-environment portability: one Zarf package deploys deterministically everywhere.
  • zarf.yaml is a version-controlled declarative manifest; zarf find-images auto-discovers image references from Helm charts.
  • zarf package create bundles Helm charts, OCI images, and Git repos with Syft SBOMs into one tar.zst.
  • A mutating webhook rewrites image and Git references on deploy, requiring no per-environment Helm value changes.
  • Per-file checksums form a Merkle tree; cosign signing chains one signature to all bundled artifacts.

About the Speaker(s)

Merijn Keppel is a Principal Consultant at TrueFullstaq focused on cloud native systems, declarative architecture, and networking.

Brandt Keller is a Staff Software Engineer at Defense Unicorns. He is a Maintainer and Technical Lead for the CNCF Security & Compliance Technical Advisory Group, a Cloud Native Ambassador, and a project maintainer within the OpenSSF.

Jessica Drost is an engineer at Isala, a hospital in the Netherlands, focused on automation, security, and observability in cloud native environments.

William Crum is a Solutions Architect at Defense Unicorns and a six-year Marine Corps veteran. He served with the Marine Innovation Unit and Defense Digital Services, driving software modernization within the Marine Corps. He co-chairs the CNCF public sector user group.

Austin Abro is a Software Engineer at Defense Unicorns and full-time maintainer of Zarf, a tool for declarative distribution of software into air-gapped environments. He previously worked at Fiat Chrysler as a full stack Java developer.