Listen to this Post
Introduction:
Scheduled container rebuilds give a false sense of security—refreshing an image every week doesn’t patch new vulnerabilities until the next cycle, leaving a window of exposure. Event‑driven rebuilds, triggered the moment a new CVE is published, eliminate that gap by rebuilding, signing, and attesting images automatically. This article breaks down how to build a “Runtime Factory” using apko, Wolfi, SLSA provenance, and automated CVE monitoring to slash attack surface from 40+ known CVEs to near zero.
Learning Objectives:
- Differentiate between scheduled rebuilds (illusion of safety) and event‑driven rebuilds (real risk reduction)
- Build minimal, shell‑free base images with apko on Wolfi that drop image size by >65% and idle memory by >80%
- Implement a nightly CVE monitor that triggers automatic rebuilds only when new vulnerabilities are detected
- Sign images and generate SLSA Level 3 provenance attestations using cosign with GitHub OIDC
- Handle unpatched CVEs transparently without failing pipelines or hiding risk
You Should Know
- Why Scheduled Rebuilds Create False Security – And How to Move to Event‑Driven Triggers
Most teams rebuild containers on a cron schedule (nightly, weekly). If a critical CVE is published 10 minutes after the last rebuild, your image remains vulnerable for up to 7 days. Attackers scan for those windows. Event‑driven monitoring flips the model: only when a new vulnerability affecting your image is released does the pipeline fire.
Step‑by‑step guide to compare and migrate:
Step 1 – Scan your current scheduled image
List all CVEs present in a container built last week:
Using Trivy (install: apt install trivy or brew install aquasecurity/trivy/trivy) trivy image yourregistry/app:latest --severity CRITICAL,HIGH
Example output showing 12 high‑severity CVEs that were not present 5 days ago.
Step 2 – Identify the delay
Calculate the difference between CVE publication date and your last rebuild:
Fetch CVE details from NVD curl -s "https://services.nvd.nist.gov/rest/json/cves/2.0?cveId=CVE-2026-3219" | jq '.vulnerabilities[bash].cve.published'
Step 3 – Set up a nightly monitor (Linux / macOS)
Create a script that checks for new CVEs against a known baseline:
!/bin/bash
nightly-cve-monitor.sh
IMAGE="yourregistry/app:latest"
BASELINE_FILE="known_cves.txt"
Scan and extract CVE IDs
trivy image --severity CRITICAL,HIGH --quiet $IMAGE | grep -oE 'CVE-[0-9]{4}-[0-9]{4,}' | sort -u > current_cves.txt
Compare with previous scan
if ! cmp -s current_cves.txt $BASELINE_FILE; then
echo "New CVEs detected. Triggering rebuild..."
curl -X POST https://api.github.com/repos/yourorg/yourrepo/dispatches \
-H "Authorization: token $GITHUB_TOKEN" \
-d '{"event_type":"rebuild-container"}'
cp current_cves.txt $BASELINE_FILE
fi
Add to crontab: `0 1 /path/to/nightly-cve-monitor.sh`
Windows equivalent (PowerShell + Task Scheduler):
$image = "yourregistry/app:latest"
$baseline = "known_cves.txt"
$current = trivy image --severity CRITICAL,HIGH $image | Select-String -Pattern "CVE-\d{4}-\d{4,}" | %{$_.Matches.Value} | Sort -Unique
$old = Get-Content $baseline
if (-not (Compare-Object $current $old -SyncWindow 0)) {
Write-Host "No new CVEs"
} else {
Write-Host "New CVEs! Trigger rebuild"
Invoke-RestMethod -Uri "https://api.github.com/repos/yourorg/yourrepo/dispatches" -Method Post -Headers @{Authorization = "token $env:GITHUB_TOKEN"}
$current | Set-Content $baseline
}
- Building Minimal Base Images with apko and Wolfi – No Shell, No Package Manager
Traditional Alpine or Debian images include shells, package managers (apk, apt), and dozens of unused libraries. Wolfi removes all of that: images contain only the exact binaries you declare. The result: image size drops from 135MB to 45MB, idle memory from 4‑6MB to under 1MB, and attack surface from 40+ CVEs to near zero.
Step‑by‑step guide:
Step 1 – Install apko
Linux / macOS curl -L https://github.com/chainguard-dev/apko/releases/latest/download/apko-linux-amd64 -o apko chmod +x apko && sudo mv apko /usr/local/bin/ Verify apko version
Step 2 – Create an apko configuration file `wolfi-python.yaml`
contents: repositories: - https://packages.wolfi.dev/os keyring: - https://packages.wolfi.dev/os/wolfi-signing.rsa.pub packages: - python-3.12 - ca-certificates - libc6-compat accounts: run-as: 1000 users: - username: appuser uid: 1000 entrypoint: command: /usr/bin/python3 -m http.server 8080 archs: - x86_64 - aarch64
Step 3 – Build the minimal image
apko build wolfi-python.yaml ghcr.io/yourorg/python-minimal:latest python-minimal.tar docker load < python-minimal.tar docker run --rm ghcr.io/yourorg/python-minimal:latest id
Output: `uid=1000(appuser) gid=65534(nobody)` – no root, no shell.
Step 4 – Compare size and CVEs
docker images | grep python trivy image ghcr.io/yourorg/python-minimal:latest | grep "Total"
Expect ~45MB and 0‑2 low severity CVEs (often false positives).
- Setting Up Nightly CVE Monitoring with Automated Rebuild Triggers
The post’s “Runtime Factory” runs nightly across all images. No new CVE? Do nothing. New CVE detected? Fire a rebuild that scans both architectures, signs the image, and publishes SLSA provenance.
Step‑by‑step using GitHub Actions:
Step 1 – Workflow to monitor CVEs (`.github/workflows/nightly-cve-monitor.yml`)
name: Nightly CVE Monitor
on:
schedule:
- cron: '0 0 ' Every night at midnight
workflow_dispatch:
jobs:
monitor:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Install Trivy
run: curl -sfL https://raw.githubusercontent.com/aquasecurity/trivy/main/contrib/install.sh | sh -s -- -b /usr/local/bin
- name: Scan current base image
id: scan
run: |
trivy image --severity CRITICAL,HIGH --format json ghcr.io/yourorg/base:latest > scan.json
CVE_LIST=$(jq -r '.Results[].Vulnerabilities[]?.VulnerabilityID' scan.json | sort -u)
echo "cves=$CVE_LIST" >> $GITHUB_OUTPUT
- name: Compare with stored baseline
id: compare
run: |
if [ -f baseline.txt ]; then
NEW_CVES=$(comm -13 <(sort baseline.txt) <(sort <<< "${{ steps.scan.outputs.cves }}"))
if [ -n "$NEW_CVES" ]; then
echo "new=true" >> $GITHUB_OUTPUT
echo "NEW_CVEs: $NEW_CVES"
fi
else
echo "new=true" >> $GITHUB_OUTPUT First run, rebuild anyway
fi
- name: Trigger rebuild if new CVEs found
if: steps.compare.outputs.new == 'true'
run: |
curl -L -X POST \
-H "Accept: application/vnd.github+json" \
-H "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}" \
https://api.github.com/repos/${{ github.repository }}/actions/workflows/rebuild.yml/dispatches \
-d '{"ref":"main"}'
- name: Update baseline
if: steps.compare.outputs.new == 'true'
run: echo "${{ steps.scan.outputs.cves }}" > baseline.txt
Step 2 – Rebuild workflow (`.github/workflows/rebuild.yml`)
Triggered by dispatcher – builds, scans both architectures, signs, and publishes:
name: Rebuild Base Image on: repository_dispatch: types: [rebuild-container] workflow_dispatch: jobs: rebuild: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Build with apko (x86_64 + aarch64) run: | apko build wolfi-config.yaml ghcr.io/yourorg/base:latest base.tar docker load < base.tar - name: Scan rebuilt image run: trivy image --severity CRITICAL,HIGH --exit-code 1 --ignore-unfixed ghcr.io/yourorg/base:latest continue-on-error: true Don't fail on unfixed CVEs - name: Sign image with cosign run: | cosign sign --key env://COSIGN_PRIVATE_KEY ghcr.io/yourorg/base:latest - name: Generate SLSA provenance uses: slsa-framework/slsa-github-generator/.github/workflows/[email protected] with: provenance-name: base-image.provenance compiled-artifacts: base.tar - name: Attest provenance run: cosign attest --predicate base-image.provenance --type slsaprovenance ghcr.io/yourorg/base:latest
Step 3 – Verify signature and provenance
cosign verify ghcr.io/yourorg/base:latest --certificate-identity-regexp "https://github.com/yourorg/." --certificate-oidc-issuer "https://token.actions.githubusercontent.com" cosign verify-attestation --type slsaprovenance ghcr.io/yourorg/base:latest
Output shows exact SHA, commit, workflow, and pipeline – any tampering fails verification.
- Handling Unpatched CVEs – The “Honest 1 CVE” Pattern
The post highlights a real‑world scenario: CVE‑2026‑3219 in pip 26.0.1 with no fix upstream. A naive pipeline would fail and create noise. A runtime factory rebuilds everything it can and stops at the boundary, then reports the unfixed CVE transparently.
Step‑by‑step to implement graceful handling:
Step 1 – Mark unfixed CVEs in Trivy
trivy image --ignore-unfixed --severity CRITICAL,HIGH yourimage:latest
Adding `–ignore-unfixed` skips CVEs without a patch version. But that hides risk. Better to allow them but alert differently.
Step 2 – Create an exception list
`unfixable-cves.txt`:
CVE-2026-3219 CVE-2025-1234
Step 3 – Scan and filter
trivy image --severity CRITICAL,HIGH --format json yourimage:latest | jq ' .Results[].Vulnerabilities[]? | select(.FixedVersion == null) | .VulnerabilityID' -r > unfixed_found.txt Block pipeline only if a CVE is not in the exception list comm -23 unfixed_found.txt unfixable-cves.txt | wc -l
If output >0, fail the build with clear message: “New unfixed CVE without exception – require upstream fix or risk acceptance.”
Step 4 – Document in attestation
Add custom predicate to SLSA provenance:
cosign attest --predicate <(cat <<EOF
{
"unfixed_cves": ["CVE-2026-3219"],
"reason": "No upstream fix available as of 2026-05-05",
"mitigation": "Network isolation, no internet access from container"
}
EOF
) --type custom ghcr.io/yourorg/base:latest
5. Hardening Cloud Deployments with Runtime Factory Outputs
Once you have near‑zero‑CVE images, deploy them securely in Kubernetes or ECS.
Step 1 – Enforce image verification with OPA/Kyverno
Kyverno policy: reject unsigned images apiVersion: kyverno.io/v1 kind: ClusterPolicy metadata: name: require-cosign-signature spec: validationFailureAction: Enforce rules: - name: check-signature match: any: - resources: kinds: - Pod verifyImages: - image: "ghcr.io/yourorg/" key: |- --BEGIN PUBLIC KEY-- ... your cosign public key ... --END PUBLIC KEY--
Step 2 – Run as non‑root (uid 1000) – verify at runtime
kubectl exec your-pod -- id Expected: uid=1000(appuser) gid=1000(appuser)
Step 3 – Monitor for drift
Use Falco to detect if a shell is added after deployment:
Falco rule - rule: Shell spawned in container condition: > spawned_process and container and proc.name in (sh, bash, dash, busybox) output: "Shell (%proc.name) spawned in container %container.id" priority: CRITICAL
What Undercode Say
- Scheduled rebuilds are security theater – They create predictable windows of exposure that attackers can map and exploit. Event‑driven rebuilds triggered by CVE publication close that gap to minutes instead of days.
- Minimal images (Wolfi/apko) are non‑negotiable – Removing shells, package managers, and unused libraries reduces the attack surface by >95%. Combined with SLSA provenance, you achieve verifiable, tamper‑proof supply chain security.
Analysis: The industry is drowning in SCA noise – thousands of CVEs, most of them irrelevant or unfixed. Ekene’s “Runtime Factory” elegantly solves two problems: it stops wasting cycles rebuilding when nothing changed, and it provides cryptographic proof of what actually runs. The hardest part – handling unpatched CVEs without failing pipelines – is where most tools give up. By rebuilding only what’s fixable and reporting the rest honestly, you move from compliance checkbox to real risk management. Expect every major container registry to offer this as a service within 18 months.
Prediction
Event‑driven rebuilds will become the default within two years. Cloud providers will bundle “auto‑remediate on CVE” into managed container services (ECR, ACR, GAR). Supply chain attacks will shift from exploiting outdated base images to attacking build pipelines directly – so SLSA Level 3+ attestations will be required for any production deployment. Organizations that still use scheduled rebuilds in 2027 will face breach rates 4x higher than those using runtime factories. The battle is moving from “do we update?” to “can we prove nothing changed besides the fix?”
▶️ Related Video (82% Match):
🎯Let’s Practice For Free:
IT/Security Reporter URL:
Reported By: Https: – Hackers Feeds
Extra Hub: Undercode MoN
Basic Verification: Pass ✅
