tldr; This first post is a deep dive into the creation of a bundle (the first class citizen of the new uds-cli). Most people reading this will get bored in 0.5s, so feel free to skip!

Bundles are ever-evolving. This post is based on the current state of the feature, but may be out of date by the time you read it.

For an overview on bundles, please read the ADR

To track feature progress, feel free to checkout the bundle PR

If you can't razzle them with brilliance...

...baffle(s) them with zarf. Enjoy this GIF of a zarf bundle create.

zarf bundle create

Bundles are like onions

Bundles are collections of Zarf OCI packages that have been merged into a single OCI image, but must still retain their individuality so that existing package deployment mechanisms can be used with minimal changes. This is to allow for both a smooth transition to bundles, but also for rapid patching of individual packages without having to rebuild an entire bundle.

If A, B, C are Zarf packages, then bundle D = A + B + C:


// Deploying all 3 individually must be the same as deploying the bundle
deploy(A) + deploy(B) + deploy(C) == deploy(D)

deploy(D) == deploy(A) + deploy(B) + deploy(C)

// Removal must also be atomic
remove(D) == remove(A) + remove(B) + remove(C)

deploy(D) + remove(A) == deploy(B) + deploy(C)

zarf-bundle.yaml schema

Bundles will follow a new YAML schema, zarf-bundle.yaml. This schema utilizes portions of the zarf.yaml schema (notably metadata and build), but focuses on orchestrating packages, not components.

metadata:
  name: bundle
  description: a bundle
  version: 0.0.1

packages:

  - repository: localhost:888/init
    ref: "###ZARF_BNDL_TMPL_INIT_VERSION###"
    optional-components:
      - git-server
    public-key: |-
        -----BEGIN PUBLIC KEY-----
        MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEGXxUhVYpuKyWNXFwjaRKiNHQcKyI
        wjoIQCI8Th5WS/Bkbmxxbxa7v20c+w9DgyeB450qsGJoaFh+uMhdbSwlCA==
        -----END PUBLIC KEY-----

  - repository: localhost:889/manifests
    ref: 0.0.1
    optional-components:
      - "*" # grab all components

bundle create

The bundle create command operates similar to the zarf package create command, with one key difference:

  • The output is only an OCI reference, not a directory bundle create <directory> -o oci://<reference>

At this time, both the source of a bundle's packages, and the resulting bundle are only stored in OCI registries. This may change in the future, but for now, it's the only supported method (pulling down into a tarball + deploying a bundle from a local tarball will be supported however).

Creation Madness

The bundle create command is a bit of a beast (src/pkg/bundler/create.go).

  1. cd into the provided directory containing the zarf-bundle.yaml file
  2. Read the zarf-bundle.yaml file into memory
  3. Template the zarf-bundle.yaml file, replacing ###ZARF_BNDL_TMPL_*### with the appropriate values
  4. Populate the build key with current build information
  5. Validate access to all of the packages in the packages key + validate package signatures w/ public key (if provided + signed)
    1. This also verifies the optional-components key, ensuring that all components specified exist, match the bundle's architecture, and expands wildcards (*, aws-*, etc)
    2. This also mutates the ref key to the full OCI reference of the package (a la localhost:888/init:v0.28.2@<sha256>...), you will see why when we get to bundle pull
  6. Build a new OCI client to the provided OCI registry, ref: <user provided>/<metadata.name>:<metadata.version>-<metadata.arch
  7. Sign the bundle (if desired)
  8. Create the bundle on the OCI registry
  9. Profit

You are doing what?

There are two ways a package gets merged into a bundle:

  • If the package is on the same registry as the bundle, the package's layers are blob mounted into the bundle's manifest

blob mounting is a very efficient means of getting data into an OCI image, as it doesn't require any data to be transferred, only performing a few HTTP requests to make the layers from one image available to another

  • If the package is on a different registry than the bundle, the package's layers are downloaded, and then uploaded to the bundle's registry. PSYCH! Why download and re-upload when we can just pipe the two requests together and stream the layers from one repository to another without ever touching the disk? thats exactly what I did

This results in a manifest that looks like the below:

{
  "schemaVersion": 2,
  "config": {
    "mediaType": "application/vnd.oci.image.config.v1+json",
    "digest": "sha256:64b63c9478c2fc5fc9f733159720d41e490dcc617c773b745568f12310d42ffb",
    "size": 153
  },
  "layers": [
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "digest": "sha256:36a8a87e199aacdf0e10be48758093e31076c93d98e7c53f4df3e8fdf69371d3",
      "size": 20869
    },
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "digest": "sha256:60e4fb5cdd71dfc11ff7fdb652fd5980ab375a5a7db7316ce75396738fab5b22",
      "size": 8327
    },
    {
      "mediaType": "application/vnd.zarf.layer.v1.blob",
      "digest": "sha256:f422ba8364518056f9c86fcf860c3bd482c778d26600e6dfe65f8e83710fb83b",
      "size": 579,
      "annotations": {
        "org.opencontainers.image.title": "zarf-bundle.yaml"
      }
    }
  ],
  "annotations": {
    "org.opencontainers.image.description": "a bundle"
  }
}

Storing the source package's manifests as layers in the bundle's manifest preserves chain of custody and allows for some fancy expansion during bundle pull (more on that later).