May 2026
This document describes how to make VMware Cloud Foundation (VCF) CLI plugins and VKS Standard Packages available in a network-isolated (air-gapped) environment. In an air-gapped deployment, the nodes and management tools have no direct access to Broadcom’s public package registries. All software must be mirrored to a local container registry and then configured as the source for addon installation. This guide uses Harbor — a CNCF-graduated, OCI-compliant container registry — as the local mirror. Harbor stores both the VCF CLI plugin inventory artifacts and the full VKS Standard Packages bundle (~409 OCI objects), and serves them to the Supervisor’s addon controller over HTTPS using its own TLS certificate.
The procedure is divided into three parts:
| Part | Summary |
|---|---|
| Part I | Download VCF CLI and its plugins on an internet-connected machine, then upload them to Harbor |
| Part II | Download the VKS Standard Packages bundle, mirror it to Harbor |
| Part III | Register Harbor with the vSphere Supervisor and redirect the addon repository to the local mirror |
A validation step at the end confirms that addon images are being pulled from Harbor and not from the internet.
The procedures and configurations in this document were validated against the following platform versions:
| Component | Version |
|---|---|
| VMware Cloud Foundation | 9.0.2.0 |
| Supervisor | v1.32.9+vmware.2-fips.vsc9.0.2.0100-25262241 |
| vSphere Kubernetes Service | 3.6.0+v1.35 |
| Kubernetes Release | v1.35.2+vmware.1-vkr.3 |
┌────────────────────────────────────┐ ┌──────────────────────────────────────────────┐
│ Internet-Connected Machine │ │ Air-Gapped Environment │
│ │ │ │
│ 1. Download VCF CLI │ │ ┌──────────────────┐ │
│ 2. vcf plugin download-bundle ────┼──tar──┼─►│ Harbor Registry │ │
│ 3. imgpkg copy -b ────────────────┼──tar──┼─►│ (local mirror) │ │
│ │ │ └──────┬───────────┘ │
└────────────────────────────────────┘ │ │ │
│ ┌──────▼───────────┐ │
│ │ vSphere │ │
│ │ Supervisor │◄── kubectl / vcf CLI │
│ └──────┬───────────┘ │
│ │ AddonRepository (override) │
│ ┌──────▼───────────┐ │
│ │ VKS Guest │ │
│ │ Cluster │ │
│ └──────────────────┘ │
└──────────────────────────────────────────────┘
Harbor is an open-source, CNCF-graduated container registry that provides OCI-compliant artifact storage, role-based access control, vulnerability scanning, and content trust over a standard HTTPS endpoint. In this deployment, Harbor is the central artifact store for all VCF and VKS software components inside the air-gapped environment.
Harbor is well-suited to this role for several reasons:
imgpkg OCI bundle containing ~409 artifacts —
not just container images, but also Helm charts, ConfigMaps, and
metadata objects. Harbor’s OCI-compliant storage layer stores and serves
all artifact types without modification./api/v2.0/ used throughout this guide to create projects
and verify artifact counts programmatically, without requiring browser
access or the Harbor UI.imgpkg or docker. This guide covers each trust
step.Two Harbor projects are created during this procedure:
| Project | Name | Visibility | Contents | Used by |
|---|---|---|---|---|
| VCF CLI Plugins | vcf-cli |
Public | VCF CLI plugin inventory OCI artifacts | VCF CLI on any workstation inside the air-gapped network |
| VKS Standard Packages | vcf-packages |
Public | Full VKS Standard Packages bundle (~409 OCI artifacts) | Supervisor addon controller; VKS cluster nodes |
Both projects are configured as public in this
guide, allowing anonymous pulls without authentication. This simplifies
the Supervisor AddonRepository configuration — no pull
secret is required. If your security policy mandates private projects,
create a Kubernetes Secret with Harbor credentials and
reference it in the AddonRepository spec under
spec.fetch.imgpkgBundle.secretRef.
After completing both mirror operations, the Harbor instance has the following repository structure:
harbor.example.com/
├── vcf-cli/
│ └── plugins/
│ ├── plugin-inventory:latest ← VCF CLI plugin index artifact
│ ├── addon/ ← addon plugin OCI artifact
│ ├── cluster/ ← cluster plugin OCI artifact
│ └── ... ← remaining plugins
│
└── vcf-packages/
└── 3.6.0-20260521/
└── vks-standard-packages:3.6.0-20260521 ← full bundle (~409 artifacts)
VCF CLI reads the plugin-inventory:latest artifact at
vcf-cli/plugins/ to enumerate available plugins. The
Supervisor addon controller reads the bundle root at
vcf-packages/3.6.0-20260521/vks-standard-packages to
enumerate available addons.
For production environments, replace the
$HARBOR_USER/$HARBOR_PASS admin credentials
used in this guide with robot accounts. Robot accounts
are scoped to specific Harbor projects, can be granted push-only or
pull-only access, and issue token-based credentials that can be rotated
independently of the admin account.
Create a robot account with push and pull access to both VCF projects via the Harbor REST API:
curl -k -u "$HARBOR_USER:$HARBOR_PASS" \
-H "Content-Type: application/json" \
-X POST "https://$HARBOR/api/v2.0/robots" \
-d '{
"name": "vcf-mirror-robot",
"duration": -1,
"description": "Push/pull access for VCF air-gapped mirroring",
"permissions": [
{
"kind": "project",
"namespace": "vcf-packages",
"access": [
{"resource": "repository", "action": "push"},
{"resource": "repository", "action": "pull"}
]
},
{
"kind": "project",
"namespace": "vcf-cli",
"access": [
{"resource": "repository", "action": "push"},
{"resource": "repository", "action": "pull"}
]
}
]
}' | jq '{name: .name, secret: .secret}'The response includes a secret field — this is the robot
account password and is shown only once. Use
robot$vcf-mirror-robot as the username (Harbor prefixes all
robot account names with robot$). Store the secret in a
vault or a chmod 600 secrets file, then substitute it for
$HARBOR_PASS throughout this guide.
Harbor’s TLS certificate is signed by a private CA in most on-premises deployments. The CA certificate must be trusted by four distinct consumers, each using a different trust mechanism:
| Consumer | Trust mechanism | Where covered |
|---|---|---|
imgpkg on the mirroring machine |
OS trust store + --registry-ca-cert-path flag |
Part II, Step 3 |
docker on the mirroring machine |
/etc/docker/certs.d/$HARBOR/ca.crt |
Part II, Step 3 |
| vSphere Supervisor | Container Registry registration in the vSphere Client UI | Part III, Step 1 |
| VKS cluster nodes (at boot) | osConfiguration.trust.additionalTrustedCAs in the
Cluster manifest |
Cluster creation guide |
All four trust steps are required. Missing any one of them produces a
TLS handshake failure at the point that consumer first contacts Harbor —
a pod ImagePullBackOff on a VKS cluster node, for example,
almost always means the node CA trust step was skipped.
Before starting, ensure the following are available:
curl, docker, tarcurl, docker, jq, Harbor registry
accessible443;
OCI artifact support must be enabled (enabled by default in Harbor
2.0+)vcf-cli and vcf-packages — will be created via
the Harbor REST API during this procedure; no manual UI setup is
requiredopenssl s_client (Step 3 of
Part II) — ensure the Harbor endpoint is reachable from the air-gapped
machine before startingvcf-cli and
vcf-packages projects (see the Robot Accounts section above
for production setup)govc (optional, used in the API
section)Set the following environment variables before running any commands. Replace the placeholder values with your actual environment details:
export HARBOR='harbor.example.com'
export HARBOR_USER='admin'
export HARBOR_PASS='<your-harbor-password>'
export SUPERVISOR_IP='<supervisor-api-endpoint-ip>'
export CLUSTER_NAMESPACE='<vsphere-namespace>'
export VCF_CLI_VSPHERE_USERNAME='administrator@vsphere.local'
export VCF_CLI_VSPHERE_PASSWORD='<your-vsphere-password>'Security note: Avoid embedding credentials directly in scripts or YAML files. Use a secrets manager or a
.envfile with restricted permissions (chmod 600) when working in a shared environment.
VCF CLI is the command-line interface for managing VCF workloads. It
uses a plugin architecture: core capabilities are provided by separately
installed plugins (such as addon, cluster,
kubernetes-release). In an air-gapped environment, the
plugin inventory must be served from your local Harbor instance rather
than from Broadcom’s public OCI registry.
Download the VCF CLI binary from Broadcom’s package repository and install it system-wide:
curl -L -o vcf-cli.tar.gz \
https://packages.broadcom.com/artifactory/vcf-distro/vcf-cli/linux/amd64/v9.0.2/vcf-cli.tar.gz
tar -xzvf vcf-cli.tar.gz
sudo install -m 0755 vcf-cli-linux_amd64 /usr/local/bin/vcf
# Verify the installation
/usr/local/bin/vcf versionVCF CLI resolves plugins from an OCI artifact called the plugin inventory. Verify which registry it currently points to:
vcf plugin source listExpected output:
NAME IMAGE
default projects.packages.broadcom.com/vcf-cli/plugins/plugin-inventory:latest
This confirms that VCF CLI currently fetches plugins from Broadcom’s public registry. The goal of this procedure is to redirect this to your local Harbor.
Before downloading, inspect the available plugins and their versions:
vcf plugin searchExpected output:
NAME DESCRIPTION LATEST
addon Add-on lifecycle management v3.6.2
cluster Kubernetes cluster operations v3.6.2
imgpkg package, distribute, and relocate your configuration and dependent oci images v9.0.2
kubernetes-release Kubernetes release operations v3.6.2
namespaces discover vsphere supervisor namespaces you have access to v9.0.0
package vcf package management v3.6.2
pais Welcome to the Private AI Services platform. v2.1.0
registry-secret registry-secret management v3.6.2
secret Secret Store Plugin for VCF CLI v9.0.2
telemetry Telemetry for VCF Workload CLI v9.0.2
vm VM Plugin for VCF Workload CLI v9.0.2
Download all plugins as a single archive. This file will be transferred to the air-gapped environment:
# Download all plugins (~6 GB)
vcf plugin download-bundle --to-tar /data/vcf-9.0.2.0/vcf-plugins-v9.0.2.tar.gz
du -h /data/vcf-9.0.2.0/vcf-plugins-v9.0.2.tar.gz
# Expected: ~6.2GAlternatively, download only specific plugins to reduce the transfer size:
vcf plugin download-bundle --plugin addon:v3.6.2 \
--to-tar /data/vcf-9.0.2.0/vcf-plugins-addon-v3.6.2.tar.gz
vcf plugin download-bundle --plugin cluster:v3.6.2 \
--to-tar /data/vcf-9.0.2.0/vcf-plugins-cluster-v3.6.2.tar.gz
vcf plugin download-bundle --plugin imgpkg:v9.0.2 \
--to-tar /data/vcf-9.0.2.0/vcf-plugins-imgpkg-v9.0.2.tar.gz
vcf plugin download-bundle --plugin kubernetes-release:v3.6.2 \
--to-tar /data/vcf-9.0.2.0/vcf-plugins-kubernetes-release-v3.6.2.tar.gzTransfer the resulting .tar.gz file(s) to the air-gapped
environment using whatever mechanism is available (USB, secure file
transfer, etc.).
Harbor organizes images into projects (similar to namespaces). Create a dedicated project for VCF CLI plugins:
HARBOR_PROJECT='vcf-cli'
curl -k -u "$HARBOR_USER:$HARBOR_PASS" \
-H "Content-Type: application/json" \
-X POST "https://$HARBOR/api/v2.0/projects" \
-d "{\"project_name\":\"$HARBOR_PROJECT\",\"public\":true}"
# Verify the project was created
curl -k -u "$HARBOR_USER:$HARBOR_PASS" \
"https://$HARBOR/api/v2.0/projects?name=$HARBOR_PROJECT"Setting the project as "public": true allows VCF CLI to
pull the plugin inventory without authenticating. Set it to
false and provide credentials in
vcf plugin source update if your security policy requires
it.
The vcf plugin upload-bundle command uses Docker’s
credential store internally. Log in before uploading:
echo "$HARBOR_PASS" | docker login "$HARBOR" -u "$HARBOR_USER" --password-stdinProduction note: In production, replace
$HARBOR_USER/$HARBOR_PASSwith a Harbor robot account (see the Robot Accounts section). The robot username format isrobot$<account-name>— for example,robot$vcf-mirror-robot. Using a robot account limits the blast radius of a credential leak: it cannot access other Harbor projects or perform administrative operations.
Push the bundle archive to the Harbor project:
vcf plugin upload-bundle \
--tar /data/vcf-9.0.2.0/vcf-plugins-v9.0.2.tar.gz \
--to-repo $HARBOR/$HARBOR_PROJECT/pluginsThis command unpacks the OCI artifacts from the tar file and pushes
them to $HARBOR/vcf-cli/plugins/, including the plugin
inventory index artifact (plugin-inventory:latest).
Update the VCF CLI configuration to resolve plugins from your local Harbor instead of Broadcom’s public registry:
vcf plugin source update default \
--uri $HARBOR/$HARBOR_PROJECT/plugins/plugin-inventory:latest
# Confirm the source was updated
vcf plugin source listExpected output (the URI now points to your Harbor):
NAME IMAGE
default harbor.example.com/vcf-cli/plugins/plugin-inventory:latest
Remove any cached plugin metadata, then install the required plugins:
# Clear cached plugin state
vcf plugin clean
# Verify the plugin list resolves from Harbor
vcf plugin searchExpected output (note the refresh message referencing your Harbor):
[i] Refreshing plugin inventory cache for "harbor.example.com/vcf-cli/plugins/plugin-inventory:latest"...
NAME DESCRIPTION LATEST
addon Add-on lifecycle management v3.6.2
cluster Kubernetes cluster operations v3.6.2
...
Install the core plugins needed for VKS cluster and addon management:
vcf plugin install addon
vcf plugin install cluster
vcf plugin install kubernetes-release
# Verify all plugins are installed
vcf plugin listExpected output:
NAME DESCRIPTION INSTALLED RECOMMENDED STATUS
addon Add-on lifecycle management v3.6.0 v3.6.0 installed
cluster Kubernetes cluster operations v3.6.0 v3.6.0 installed
kubernetes-release Kubernetes release operations v3.6.0 v3.6.0 installed
namespaces discover vsphere supervisor namespaces v9.0.0 v9.0.0 installed
package VCF Package management v3.6.0 v3.6.0 installed
registry-secret Registry secret management v3.6.0 v3.6.0 installed
...
The VKS Standard Packages repository is a large OCI bundle (~13 GB) that contains all container images required by VKS add-ons: cert-manager, Cilium, Contour, Prometheus, Velero, and others. In an air-gapped environment, this bundle must be mirrored to Harbor so that the Supervisor’s addon controller can pull images locally.
The tool used to copy OCI bundles is imgpkg, part of the
Carvel toolchain.
imgpkg is part of the Carvel suite of Kubernetes
packaging tools. Install it on the machine where you will perform the
mirroring:
# Download the Carvel installation script
wget -O /tmp/install.sh https://carvel.dev/install.sh
# Make it executable and run as root
chmod +x /tmp/install.sh
sudo /tmp/install.shVerify the installation:
imgpkg versionConsult the VKS Standard Packages Release Notes to find the correct bundle version for your VCF release.
This guide uses version 3.6.0+20260521. Set the version
as a variable:
VKS_VERSION="3.6.0-20260521"
VKS_BUNDLE="projects.packages.broadcom.com/vsphere/supervisor/vks-standard-packages/${VKS_VERSION}/vks-standard-packages:${VKS_VERSION}"Run imgpkg copy with --to-tar on an
internet-connected machine. This downloads the entire bundle — all
images and OCI metadata — into a single portable tar archive:
imgpkg copy \
-b $VKS_BUNDLE \
--to-tar /data/vcf-9.0.2.0/vcf-standard-packages-${VKS_VERSION}.tar
du -h /data/vcf-9.0.2.0/vcf-standard-packages-${VKS_VERSION}.tar
# Expected: ~13 GBTransfer this file to the air-gapped environment.
Before imgpkg can push to Harbor over HTTPS, the Harbor
CA certificate must be trusted by both the OS and Docker. This is
required whenever Harbor uses a self-signed or private CA.
mkdir -p /tmp/harbor-ca
openssl s_client -showcerts \
-connect $HARBOR:443 \
-servername $HARBOR </dev/null 2>/dev/null \
| awk '/BEGIN CERTIFICATE/,/END CERTIFICATE/ {print}' \
> /tmp/harbor-ca/$HARBOR.crtopenssl x509 -in /tmp/harbor-ca/$HARBOR.crt -noout -subject -issuer -datesReview the output to confirm the certificate belongs to your Harbor instance and has not expired.
Docker uses per-registry certificate directories under
/etc/docker/certs.d/. Create the directory and place the CA
certificate there:
sudo mkdir -p /etc/docker/certs.d/$HARBOR
sudo cp /tmp/harbor-ca/$HARBOR.crt /etc/docker/certs.d/$HARBOR/ca.crt
sudo systemctl restart dockerThe OS trust store is used by tools like curl and
imgpkg (which uses Go’s TLS stack). Add the certificate to
the system trust store:
sudo cp /tmp/harbor-ca/$HARBOR.crt /etc/pki/ca-trust/source/anchors/$HARBOR.crt
sudo update-ca-trustOn Debian/Ubuntu systems, use:
sudo cp /tmp/harbor-ca/$HARBOR.crt /usr/local/share/ca-certificates/$HARBOR.crt
sudo update-ca-certificatesThe vcf-packages project will hold the VKS Standard
Packages bundle under a versioned path. The project is created as public
so the Supervisor addon controller can pull images without embedding
credentials in the AddonRepository resource.
HARBOR_PROJECT='vcf-packages'
curl -k -u "$HARBOR_USER:$HARBOR_PASS" \
-H "Content-Type: application/json" \
-X POST "https://$HARBOR/api/v2.0/projects" \
-d "{\"project_name\":\"$HARBOR_PROJECT\",\"public\":true}"
# Verify project creation
curl -k -u "$HARBOR_USER:$HARBOR_PASS" \
"https://$HARBOR/api/v2.0/projects?name=$HARBOR_PROJECT"Use imgpkg copy with --from-tar to push the
bundle from the tar archive to your Harbor repository. The
--registry-ca-cert-path flag explicitly passes the Harbor
CA so imgpkg trusts the HTTPS connection:
imgpkg copy \
--tar /data/vcf-9.0.2.0/vcf-standard-packages-${VKS_VERSION}.tar \
--to-repo $HARBOR/$HARBOR_PROJECT/${VKS_VERSION}/vks-standard-packages \
--registry-ca-cert-path /tmp/harbor-ca/$HARBOR.crtNote: This push operation may take 30–60 minutes depending on Harbor’s disk speed and available CPU, as it processes ~409 OCI artifacts.
Confirm that the repository and artifacts were created successfully:
curl -ksu "$HARBOR_USER:$HARBOR_PASS" \
"https://$HARBOR/api/v2.0/projects/$HARBOR_PROJECT/repositories" | jqExpected output (example with two versions present):
[
{
"artifact_count": 409,
"creation_time": "2026-05-30T21:17:52.937Z",
"id": 79,
"name": "vcf-packages/3.6.0-20260521/vks-standard-packages",
"project_id": 8,
"pull_count": 0,
"update_time": "2026-05-30T21:17:52.937Z"
}
]An artifact_count of ~409 confirms the full bundle was
transferred successfully.
The vSphere Supervisor must be configured to trust Harbor before the addon controller can pull images from it. This is done in two sub-steps:
AddonRepository resource that points to
the Harbor mirrorNavigate to the vSphere Client and add Harbor as a trusted container registry for the Supervisor:
Supervisor Management
→ Supervisors
→ [Select your Supervisor]
→ Configure
→ Container Registries
→ Add
Fill in the form with the following values:
| Field | Value |
|---|---|
| Name | harbor-<your-org-name> (e.g.,
harbor-vcf-example) |
| Registry Host URL | harbor.example.com |
| CA Certificate | Full PEM contents of /tmp/harbor-ca/$HARBOR.crt |
| Username / Password | Harbor admin credentials (only required if the project is private) |
Click Save. The Supervisor will now trust TLS certificates signed by the Harbor CA.
Use VCF CLI to establish a Kubernetes context pointing to the
Supervisor’s API server. This allows kubectl commands to
manage Supervisor-level resources such as AddonRepository
and AddonRepositoryInstall.
export CLUSTER_NAMESPACE='<your-vsphere-namespace>'
export SUPERVISOR_IP='<supervisor-api-ip>'
export VCF_CLI_VSPHERE_USERNAME='administrator@vsphere.local'
export VCF_CLI_VSPHERE_PASSWORD='<your-vsphere-password>'
# Remove any existing supervisor context and create a fresh one
vcf context delete supervisor
vcf context create supervisor \
--endpoint ${SUPERVISOR_IP} \
--username ${VCF_CLI_VSPHERE_USERNAME} \
--insecure-skip-tls-verify \
--type k8s \
--auth-type basic
vcf context use supervisorNote: The
--insecure-skip-tls-verifyflag is used here because many Supervisor endpoints use a self-signed certificate. In production, provide the Supervisor CA certificate instead.
The Supervisor uses an AddonRepository custom resource
to determine where addon packages are fetched from. By default, it
points to Broadcom’s public registry. You will create a new
AddonRepository that points to Harbor, then patch the
existing AddonRepositoryInstall to use it.
REGISTRY="$HARBOR"
HARBOR_PROJECT="vcf-packages"
VKS_VERSION="3.6.0-20260521"
VKS_STANDARD_REPO="$HARBOR_PROJECT/${VKS_VERSION}/vks-standard-packages"
IMAGE_URL="${REGISTRY}/${VKS_STANDARD_REPO}:${VKS_VERSION}"
echo "Using image: $IMAGE_URL"Expected output:
Using image: harbor.example.com/vcf-packages/3.6.0-20260521/vks-standard-packages:3.6.0-20260521
The AddonRepository resource tells the Supervisor’s
addon controller where to fetch the package bundle. The
annotations field includes a JSON payload that records
which version and packages are available — this is read by the addon
controller to populate the available addon list.
cat > /tmp/default-addonrepository-override.yaml <<EOF
apiVersion: addons.kubernetes.vmware.com/v1alpha1
kind: AddonRepository
metadata:
name: default-addonrepository-override
namespace: vmware-system-vks-public
annotations:
addons.kubernetes.vmware.com/package-offerings: |
{
"repositoryVersion": "${VKS_VERSION}",
"packages": {}
}
spec:
fetch:
imgpkgBundle:
imageURL: ${IMAGE_URL}
targetRepositoryName: standard-packages
version: ${VKS_VERSION}
EOF
kubectl apply -f /tmp/default-addonrepository-override.yamlThe AddonRepositoryInstall resource
(default-addon-repo-install) controls which
AddonRepository is active. Patch it to point to the
override and add an annotation that prevents VKS-managed automation from
reverting your change:
NEW_REPO="default-addonrepository-override"
NAMESPACE="vmware-system-vks-public"
kubectl patch addonrepositoryinstall default-addon-repo-install \
-n "$NAMESPACE" \
--type merge \
-p "{
\"metadata\": {
\"annotations\": {
\"addons.kubernetes.vmware.com/skip-vks-managed-addon-repository-update\": \"\"
}
},
\"spec\": {
\"addonRepositoryRef\": {
\"name\": \"$NEW_REPO\",
\"namespace\": \"$NAMESPACE\"
}
}
}"Important: The annotation
skip-vks-managed-addon-repository-updateis required. Without it, the VKS platform automation will periodically overwrite your patch and revert theaddonRepositoryRefback to the default Broadcom-hosted repository.
kubectl get addonrepository default-addonrepository-override \
-n vmware-system-vks-publicExpected output:
NAME TARGETREPOSITORY VERSION AGE
default-addonrepository-override standard-packages 3.6.0-20260521 19h
kubectl get addonrepository default-addonrepository-override \
-n vmware-system-vks-public \
-o jsonpath='{.spec.fetch.imgpkgBundle.imageURL}{"\n"}'Expected output:
harbor.example.com/vcf-packages/3.6.0-20260521/vks-standard-packages:3.6.0-20260521
kubectl get addonrepositoryinstall default-addon-repo-install \
-n vmware-system-vks-publicExpected output:
NAME ADDONREPOSITORY AGE
default-addon-repo-install default-addonrepository-override 31d
vcf addon repository listExpected output (Harbor-hosted repository is now listed):
NAME NAMESPACE SOURCE
default-addonrepository-3.5.0-20251218 vmware-system-vks-public projects.packages.broadcom.com/...
default-addonrepository-3.6.0-20260416 vmware-system-vks-public projects.packages.broadcom.com/...
default-addonrepository-override vmware-system-vks-public harbor.example.com/vcf-packages/3.6.0-20260521/vks-standard-packages:3.6.0-20260521
Confirm the Supervisor can enumerate available addons from the local repository:
vcf addon available list
kubectl get addons -AExpected output (showing addons sourced from the override repository):
NAMESPACE NAME AGE
vmware-system-vks-public ako 31d
vmware-system-vks-public antrea 31d
vmware-system-vks-public cert-manager 31d
vmware-system-vks-public cilium 18h
vmware-system-vks-public contour 31d
vmware-system-vks-public gatekeeper 18h
vmware-system-vks-public helm-controller 18h
vmware-system-vks-public prometheus 31d
vmware-system-vks-public velero 31d
...
Install the cert-manager addon on a test cluster and
verify that all container images are pulled from Harbor.
vcf addon available list cert-managerExpected output:
NAMESPACE ADDONNAME VERSION ADDON-RELEASE-NAME PACKAGE
vmware-system-vks-public cert-manager 1.18.2+vmware.2-vks.2 cert-manager.kubernetes.vmware.com.1.18.2-vmware.2-vks.2 ...
vmware-system-vks-public cert-manager 1.19.4+vmware.1-vks.1 cert-manager.kubernetes.vmware.com.1.19.4-vmware.1-vks.1 ...
export CLUSTER_NAME='<your-test-cluster-name>'
vcf addon install create cert-manager \
--cluster-name $CLUSTER_NAME \
--namespace $CLUSTER_NAMESPACE \
--verbose 9 -yvcf addon install list \
--cluster-name $CLUSTER_NAME \
--namespace $CLUSTER_NAMESPACEExpected output:
ADDONNAME NAMESPACE ADDONRELEASENAME PAUSED READY DELETE/UPGRADE
cert-manager <your-namespace> cert-manager.kubernetes.vmware.com.1.19.4-vmware.1-vks.1 false True Allowed
Switch your kubeconfig to the guest cluster, then inspect the image references for the cert-manager pods:
kubectl get pods -n cert-manager \
-o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{range .spec.containers[*]}{.image}{" "}{end}{"\n"}{end}' \
| column -t -N POD,IMAGEExpected output — all images reference your Harbor registry:
POD IMAGE
cert-manager-85c5d5d754-2fsp8 harbor.example.com/vcf-packages/3.6.0-20260521/vks-standard-packages@sha256:92bc...
cert-manager-cainjector-94cf76d75-vnwjr harbor.example.com/vcf-packages/3.6.0-20260521/vks-standard-packages@sha256:1581...
cert-manager-webhook-6f85f6c47b-tccfx harbor.example.com/vcf-packages/3.6.0-20260521/vks-standard-packages@sha256:72ad...
If all image references point to harbor.example.com, the
air-gapped configuration is complete and working correctly.
vcf addon install delete cert-manager \
--cluster-name $CLUSTER_NAME \
--namespace $CLUSTER_NAMESPACE -y| Symptom | Likely Cause | Resolution |
|---|---|---|
vcf plugin search shows no plugins |
Plugin source still points to Broadcom; local Harbor unreachable | Run vcf plugin source list and verify the URI points to
Harbor |
imgpkg copy fails with TLS error |
Harbor CA not trusted | Re-run Step 3.3 and 3.4 and verify update-ca-trust
completed |
kubectl apply returns “image pull backoff” |
Harbor project is private or Docker login expired | Re-run docker login $HARBOR |
addonrepositoryinstall reverts to Broadcom source |
Missing skip-vks-managed-addon-repository-update
annotation |
Re-apply the patch in Step 3.3 |
artifact_count in Harbor API is 0 |
imgpkg copy failed partway through |
Re-run the imgpkg copy --from-tar command |
Harbor returns 401 Unauthorized when pushing |
Robot account lacks push access to the project, or credentials are wrong | Verify the robot account has repository:push permission
on the target project; confirm the username format is
robot$<name> |
Harbor returns 403 Forbidden when creating a
project |
The authenticated user is not a Harbor admin | Use admin credentials (or a robot account with system-level create permission) for project creation; a project-scoped robot cannot create new projects |
VKS cluster nodes fail with ImagePullBackOff after
cluster creation |
Harbor CA not injected into cluster nodes | Verify osConfiguration.trust.additionalTrustedCAs is
set in the Cluster manifest and the referenced Secret exists; the CA
trust step in the cluster manifest is separate from the Supervisor CA
registration |
imgpkg copy succeeds but artifact_count is
lower than expected (< 409) |
Harbor storage quota exceeded, or push interrupted | Check Harbor project storage quota
(Harbor UI → Projects → vcf-packages → Configuration);
increase the quota or set it to unlimited, then re-run the push |
| vSphere Supervisor cannot pull from Harbor despite Container Registry being registered | Supervisor node CA trust not propagated | After registering the Harbor CA in the vSphere Client, wait 2–3 minutes for the Supervisor to sync the CA; check the Container Registry status in the UI for any error messages |
| Name | URL |
|---|---|
| VCF CLI Documentation | https://techdocs.broadcom.com/us/en/vmware-cis/vcf/vcf-service-administration-and-development/9-0.html |
| VKS Standard Packages Release Notes | https://techdocs.broadcom.com/us/en/vmware-cis/vcf/vcf-service-administration-and-development/9-0/release-notes/vks-standard-packages-release-notes.html |
| Harbor Documentation | https://goharbor.io/docs/ |
| vCenter Namespace Management APIs | https://developer.broadcom.com/xapis/vsphere-automation-api/latest/vcenter-namespace-management/ |