This workflow creates two VKS clusters on an existing Supervisor using one shared Terraform module. Each cluster has its own configuration, provider lock file, saved plan and Terraform state. Run the procedure for both directories to create both clusters.
Target VCF release:
9.1.1.0-545627-2026-09.
| Setting | DEV | QA |
|---|---|---|
| Terraform directory | dev-cluster-01 |
qa-cluster-02 |
| Cluster name | dev-cluster-01 |
qa-cluster-02 |
| Supervisor namespace | dev-namespace |
qa-namespace |
| Control-plane nodes | 3 | 3 |
| Worker nodes | 3 | 3 |
| Containerd volume per worker | 30Gi |
30Gi |
These are desired settings from the configuration reviewed on
2026-10-22, not live deployment status. The QA directory and Cluster are
both named qa-cluster-02.
Repository: github.com/jmnicolescu/terraform-vks
Access the Terraform projects, deployment guides, and captured output files in this repository.
| Operation | Configuration or command | Completion criterion |
|---|---|---|
| Create DEV and QA | Plan and apply each root separately | Both workload APIs accessible; each cluster has 3 ready control-plane nodes and 3 ready workers |
| Upgrade Kubernetes | Change kubernetes_version, then plan and apply each
root |
Supported target release reconciled; all nodes ready at the target version; workloads healthy |
| Scale workers | Change worker_count from 3 to
5 in each root |
Each cluster has 3 ready control-plane nodes and 5 ready workers |
| Upgrade Terraform providers | Deliberately run init -upgrade in each root |
Reviewed lock-file changes, successful validation and reviewed plans |
IMPORTANT: An active network connection to the Supervisor API is required. Before running the deployment workflow, the supervisor context must exist, be available, and authenticate successfully to the intended Supervisor.
vcf), and kubectl,
and make them available in PATH.From the repository root, check whether the context already exists:
vcf context list
kubectl config get-contexts supervisorIf the supervisor context does not exist, review and
modify utilities/01-create-supervisor-context.sh
for your environment before running it:
| Script setting | What to review or change |
|---|---|
SUPERVISOR_API_ENDPOINT |
Set the intended Supervisor API address, reachable from this workstation. |
VCF_CLI_VSPHERE_USERNAME |
Set the vSphere SSO account authorized to access that Supervisor. |
VCF_CLI_VSPHERE_PASSWORD |
Review the credential handling. The current script assigns this
variable but does not export it or pass it to vcf; do not
assume that changing it supplies the CLI password. Complete
authentication when prompted, and do not commit credentials. |
CONTEXT_NAME |
Keep supervisor to match the VKS Terraform roots, or
update their supervisor_context settings and the
verification commands consistently. |
The script checks for vcf and kubectl,
attempts to delete the existing local context named
supervisor, creates it with the configured endpoint and SSO
username, activates it, lists VCF contexts, and runs
kubectl get nodes. It changes local connection
configuration; it does not create the Supervisor infrastructure or
deploy namespaces or VKS clusters. Context creation currently uses
--insecure-skip-tls-verify; review that lab TLS setting
before use.
Run the reviewed script from the repository root:
bash utilities/01-create-supervisor-context.shIf the context already exists and points to the correct Supervisor, activate it without recreating it:
vcf context use supervisorvcf context list
kubectl config get-contexts supervisor
kubectl config use-context supervisor
kubectl get nodesProceed only when supervisor points to the
intended endpoint and the explicit node query succeeds. The
script currently continues after command failures and prints its final
SUCCESS message unconditionally; that message alone does
not prove connectivity. If authentication fails, refresh the Supervisor
login. If the endpoint is unreachable, restore network/VPN access and
check the configured address. If access is forbidden, resolve the
account permissions before continuing.
The VKS Terraform roots use the authenticated supervisor
context from ~/.kube/config by default and require live
Supervisor access for planning and applying. The namespace Terraform
providers authenticate separately to vCenter; this Supervisor connection
requirement prepares the documented workflow and Kubernetes
verification, while their vCenter connection remains independently
required.
hashicorp/kubernetes ~> 3.0 and
hashicorp/local ~> 2.5; their lock files select versions
3.4.0 and 2.9.1, respectively.cluster.x-k8s.io/v1beta2 Cluster CRD. Planning and applying
both require access to the Supervisor API.~/.kube/config and the explicit context
supervisor; verify the endpoint and TLS settings before
use.dev-namespace and qa-namespace,
with the required VM/storage class assignments, compatible release and
ClusterClass, network configuration and sufficient quota for both
clusters.additional_trusted_cas = []. If you enable the commented
example, create harbor-user-trusted-ca-secret with key
harbor-ca in each affected namespace first. Terraform
references these Secrets; it does not create them.This README is in the repository root, alongside
terraform-create-vks-cluster/:
terraform-create-vks-cluster/
├── dev-cluster-01/
│ ├── main.tf
│ └── .terraform.lock.hcl
├── qa-cluster-02/
│ ├── main.tf
│ └── .terraform.lock.hcl
└── modules/vks-cluster/
├── main.tf
├── variables.tf
├── versions.tf
└── outputs.tf
Edit deployment settings in DEV main.tf and QA main.tf. Terraform reads these root settings
directly; it does not source the original lab configuration file
00-vcf9-env.sh or use the generated YAML manifests as
inputs.
Both roots currently use:
| Input | Configured value |
|---|---|
kubernetes_version |
v1.35.6---vmware.2-vkr.3 |
cluster_class |
builtin-generic-v3.7.0 |
cluster_class_namespace |
vmware-system-vks-public |
vm_class |
best-effort-medium |
storage_class |
vsan-default-storage-policy |
pod_cidrs |
["10.95.0.0/16"] |
service_cidrs |
["10.96.0.0/12"] |
containerd_volume_capacity |
30Gi, mounted at /var/lib/containerd |
Confirm the CIDRs fit the network design for each cluster. The module validates CIDR syntax, not overlap or platform compatibility. Three workers request 90 GiB of containerd storage per cluster, excluding operating-system disks and other storage.
Each root calls ../modules/vks-cluster and passes its
configured kubernetes.supervisor provider to the module.
The kubeconfig supplies authentication and the API endpoint. The shared
module builds one Cluster object with Antrea, Photon OS annotations,
certificate rotation, pod security settings and one worker pool named
<cluster-name>-np01.
See variables.tf for all inputs and validation. Changes to the shared module affect future plans for both clusters; applying one root operates only on that root’s state. Never copy state or saved plans between directories.
| Provider | Constraint / selected version | Responsibility |
|---|---|---|
hashicorp/kubernetes |
~> 3.0 / 3.4.0 |
Applies the VKS Cluster custom resource to the Supervisor through
kubernetes_manifest |
hashicorp/local |
~> 2.5 / 2.9.1 |
Writes the desired Cluster YAML through local_file |
The selected versions above come from both local
.terraform.lock.hcl files, reviewed on 2026-10-22. They
describe this checkout, not a claim that these are the latest available
versions. The constraints allow compatible releases below the next major
version; the lock files preserve the exact selections and checksums.
Keep the lock files with the configuration. Routine
terraform init reuses those selections;
init -upgrade deliberately reselects providers within the
constraints. See HashiCorp’s dependency lock-file
documentation
and init
reference.
The Kubernetes provider is appropriate because this workflow manages
a Kubernetes custom resource exposed by the Supervisor. VKS controllers
turn its topology into virtual machines and workload nodes. No vSphere
or VCF Automation Terraform provider is configured here. The root’s
kubernetes.supervisor alias supplies an explicit kubeconfig
context to the shared module, preventing reliance on whichever context
happens to be active in the shell.
kubernetes_manifest needs the resource schema during
planning. The Supervisor, VKS service and Cluster CRD must therefore
exist before running this workflow. They cannot be bootstrapped by these
roots. See HashiCorp’s custom-resource
workflow.
| Decision in the implementation | Rationale and operational consequence |
|---|---|
| One shared module, two independent roots | Keeps the manifest structure consistent while allowing separate plans, state and change windows. Shared-module edits must be reviewed against both roots. |
| Local state; no backend block | Simple for this lab, but state must be protected and backed up. Serialize operations on each root. A team workflow should deliberately configure a backend with access control, locking and recovery before concurrent use. |
| One Cluster resource per root | VKS owns derived Machines and infrastructure. Change the Cluster topology through Terraform rather than editing generated Machines or MachineDeployments. |
| Three control-plane nodes and one worker pool | Separates control-plane capacity from the independently configurable worker count. This module does not model multiple worker pools or availability-zone placement. |
| Explicit release, ClusterClass, VM class and storage class | Makes desired settings reviewable. Input validation does not establish installed-platform compatibility or available capacity. |
Server-side field manager terraform-vks-cluster,
force_conflicts = false |
Field-ownership conflicts stop the apply instead of forcibly taking fields from another manager. Investigate the competing owner before changing ownership. |
| Computed annotations, labels, topology variables and version | Accommodates Supervisor admission defaults and release normalization. Compare the requested values with the reconciled object: computed fields permit server-returned differences and are not a readiness check. |
| Explicit pod-security and certificate settings | Current policy audits at restricted and enforces/warns
at privileged, with deactivated = false; it
does not enforce the restricted profile. Certificate rotation is enabled
with a 90-day renewal lead time. Review these choices against workload
requirements. |
| Antrea and Photon OS selection | Preserves the lab topology; this module does not expose a general CNI or operating-system selector. |
Separate 30Gi containerd volume on each worker |
Keeps container image/runtime storage configurable independently of worker count. Changing volume capacity is a separate lifecycle operation requiring platform validation. |
| Optional CA Secret references | Allows additional trust without embedding certificate contents in Terraform configuration. Secrets remain externally managed. |
| Generated YAML from the same desired manifest | Provides a readable deployment artifact without maintaining a second manifest definition. It is neither a backup of live status nor an independent deployment owner. |
The field-management and computed-field behavior is described in the
official kubernetes_manifest
reference.
Local artifact behavior is documented in the local_file
reference.
Registry links may show newer documentation than the locally locked
versions; select the matching version when checking version-specific
behavior.
Start each command block containing a
cd terraform-create-vks-cluster/... command from the
repository root (terraform-vks/). Enter the selected
cluster directory before running Terraform. Blocks without a
cd continue in the directory selected by the preceding
block. When starting a different procedure, return to the repository
root (terraform-vks/) first.
umask 077
kubectl config get-contexts
kubectl config use-context supervisor
kubectl get namespaces dev-namespace qa-namespace
kubectl get crd clusters.cluster.x-k8s.ioIf you configure a different kubeconfig or Supervisor context in either root, use the same connection for its kubectl checks. If enabling additional CA trust, select the Supervisor context and check the referenced Secret in both namespaces:
kubectl config use-context supervisor
kubectl -n dev-namespace get secret harbor-user-trusted-ca-secret
kubectl -n qa-namespace get secret harbor-user-trusted-ca-secretDo not run the Bash deployment scripts as Terraform prerequisites: they submit infrastructure changes themselves. If either Cluster already exists outside its Terraform state, follow Adopt an existing cluster before planning.
cd terraform-create-vks-cluster/dev-cluster-01
terraform init
terraform validate
terraform plan -out=vks.tfplanReview the plan. For a fresh deployment, it should create
dev-namespace/dev-cluster-01 and its local YAML file.
Confirm three control-plane nodes and three workers, then apply the
saved plan:
terraform apply vks.tfplanComplete the DEV readiness checks in the next section before
proceeding to QA. Return to the repository root
(terraform-vks/) before the following block.
cd terraform-create-vks-cluster/qa-cluster-02
terraform init
terraform validate
terraform plan -out=vks.tfplanReview the plan. For a fresh deployment, it should create
qa-namespace/qa-cluster-02 and its local YAML file. Confirm
three control-plane nodes and three workers, then apply the saved
plan:
terraform apply vks.tfplanEach saved-plan apply executes the reviewed plan without a further approval prompt. Generate a fresh plan after changing configuration. Protect state and plan files, and retain both provider lock files.
Successful apply means Terraform applied the Cluster object. The module does not wait for workload readiness; Supervisor controllers provision the infrastructure asynchronously.
kubectl config use-context supervisor
kubectl -n dev-namespace get clusters.cluster.x-k8s.io dev-cluster-01 -o yaml
kubectl -n dev-namespace get machines.cluster.x-k8s.io -l cluster.x-k8s.io/cluster-name=dev-cluster-01
kubectl -n qa-namespace get clusters.cluster.x-k8s.io qa-cluster-02 -o yaml
kubectl -n qa-namespace get machines.cluster.x-k8s.io -l cluster.x-k8s.io/cluster-name=qa-cluster-02For each cluster, confirm current Available=True
conditions, three ready control-plane nodes and three ready workers.
Follow Connect to
the VKS clusters to establish each workload context and verify API
access and node readiness. Object existence and generated YAML alone do
not establish readiness.
After the Supervisor reports the target cluster as available, create
its local access context and verify the workload API. Complete the DEV
procedure for dev-cluster-01 and the QA procedure for
qa-cluster-02 independently. Run the following scripts from
the repository root (terraform-vks/), not
from a Terraform deployment directory.
IMPORTANT: Maintain an active network connection to the Supervisor before running either context script. The workstation must also be able to reach the selected workload cluster API to use kubectl after authentication.
Bash, VCF CLI (vcf), and kubectl must be
available in PATH. First establish and verify Supervisor
access using Requirements: Supervisor connection. Use a vSphere
SSO account authorized to access the target workload cluster.
Review the selected script before running it. Set
SUPERVISOR_API_ENDPOINT to the Supervisor address and
VCF_CLI_VSPHERE_USERNAME to the intended SSO account.
Verify the namespace and cluster settings against the table below.
| Environment | Script | CLUSTER_NAMESPACE |
CLUSTER_NAME |
Context activated by the script |
|---|---|---|---|---|
| DEV | 02-create-dev-cluster-01-context.sh | dev-namespace |
dev-cluster-01 |
dev-cluster-01:dev-cluster-01 |
| QA | 03-create-qa-cluster-02-context.sh | qa-namespace |
qa-cluster-02 |
qa-cluster-02:qa-cluster-02 |
Both scripts currently use --insecure-skip-tls-verify;
review this lab TLS setting for your environment. They assign
VCF_CLI_VSPHERE_PASSWORD locally but do not export it or
pass it to the CLI. Complete authentication when prompted; do not assume
the assignment supplies the password, and do not commit credentials.
Each script performs the same sequence for its configured cluster:
vcf and kubectl are installed
and available in PATH.vcf context create with the Supervisor endpoint,
SSO username, workload namespace, and workload cluster name, using
Kubernetes/basic authentication.vcf context use
with the cluster-name:cluster-name identifier shown
above.kubectl get nodes
against the active Kubernetes context.These utilities configure local access to existing clusters. They do
not create, scale, or delete VKS clusters, and they do not modify
Terraform state. Activating a workload context changes the destination
of subsequent kubectl commands.
From the repository root, create and activate the DEV context:
bash utilities/02-create-dev-cluster-01-context.shConfirm the selected context, then query DEV explicitly:
vcf context list
kubectl config current-context
kubectl config use-context dev-cluster-01:dev-cluster-01
kubectl get nodes -o wide
kubectl get pods -AThe active context should be
dev-cluster-01:dev-cluster-01. For the supplied
three-control-plane, three-worker configuration, verify that all six
nodes report Ready. Review pod status and investigate
unexpected pending, failed, or restarting workloads.
From the repository root, create and activate the QA context:
bash utilities/03-create-qa-cluster-02-context.shConfirm the selected context, then query QA explicitly:
vcf context list
kubectl config current-context
kubectl config use-context qa-cluster-02:qa-cluster-02
kubectl get nodes -o wide
kubectl get pods -AThe active context should be
qa-cluster-02:qa-cluster-02. For the supplied
configuration, verify that all six nodes report Ready and
review pod status. QA becomes the active context after this procedure;
select the matching context with kubectl config use-context
before checking each cluster.
Once the contexts exist, activate the required target with the corresponding command:
| Target | Command |
|---|---|
| DEV workload cluster | vcf context use dev-cluster-01:dev-cluster-01 |
| QA workload cluster | vcf context use qa-cluster-02:qa-cluster-02 |
| Supervisor | vcf context use supervisor |
Use the Supervisor context for Cluster and Machine resource checks, and the workload context for that cluster’s nodes, pods, and applications. The Terraform roots continue to use their explicitly configured Supervisor context.
Connection success requires a successful query against the
intended workload context. Both scripts currently continue
after command failures and print a final SUCCESS message
unconditionally. If a script reports an error, inspect the context and
explicit node-query results before proceeding. Resolve authentication,
endpoint reachability, or access permissions as indicated by the error;
if provisioning is incomplete, return to the Supervisor readiness
checks.
Each root exposes cluster_name, namespace,
manifest_yaml and manifest_file. Apply writes
the desired manifest automatically:
dev-cluster-01/output-YYYY-MM-DD/create-dev-cluster-01.yaml
qa-cluster-02/output-YYYY-MM-DD/create-qa-cluster-02.yaml
The date is the plan’s UTC date. Files use permission
0600, and newly created output directories use
0700. The YAML contains desired configuration and CA Secret
references, without runtime status or Secret contents.
From the repository root (terraform-vks/), inspect each
path in its own cluster directory:
cd terraform-create-vks-cluster/dev-cluster-01
terraform output -raw manifest_file
cd ../qa-cluster-02
terraform output -raw manifest_fileTerraform manages one current YAML file per root. A later plan with a different date can replace it, and destroy removes the tracked file. These directories are not permanent archives. Keep one lifecycle owner per Cluster; do not independently apply generated YAML to a Terraform-managed Cluster.
A Kubernetes release upgrade, worker scaling and a Terraform provider
upgrade are different changes. Plan them separately so that each result
can be verified. Start each procedure from the repository root
(terraform-vks/), enter the selected cluster directory as
shown, and use its existing state. Apply-only blocks continue in the
directory used to create the plan.
Before either a release upgrade or scaling:
Example state snapshots for already managed clusters:
cd terraform-create-vks-cluster/dev-cluster-01
umask 077
CHANGE_TIME=$(date -u +%Y-%m-%d-%H-%M-%S)
mkdir -m 700 "state-backup-${CHANGE_TIME}"
terraform state pull > "state-backup-${CHANGE_TIME}/dev.tfstate"
cd ../qa-cluster-02
mkdir -m 700 "state-backup-${CHANGE_TIME}"
terraform state pull > "state-backup-${CHANGE_TIME}/qa.tfstate"Check each command succeeds and the snapshots are nonempty. Keep these files private and outside version control. Never restore or push an older state merely to undo an infrastructure change.
The current configured release is
v1.35.6---vmware.2-vkr.3. No next release is prescribed
here: select an exact release available to this Supervisor and supported
from the current release by the installed VKS service. Confirm its OS
image availability, upgrade path and compatible ClusterClass before
editing. Do not infer compatibility from version ordering or assume that
skipping minor versions is supported.
Broadcom documents the VKS release upgrade
workflow
and versioned ClusterClass
updates.
These VCF 9.0 references provide platform background; confirm the
installed VCF 9.1.1/VKS release’s requirements. This module uses
cluster.x-k8s.io/v1beta2 and
spec.topology.classRef; do not copy legacy v1alpha3 field
paths into it.
Inspect the current objects and discover the release resources exposed by the Supervisor:
kubectl config use-context supervisor
kubectl api-resources
kubectl -n vmware-system-vks-public get clusterclasses.cluster.x-k8s.io
kubectl -n dev-namespace get clusters.cluster.x-k8s.io dev-cluster-01 -o yaml
kubectl -n qa-namespace get clusters.cluster.x-k8s.io qa-cluster-02 -o yamlUse the discovered release resource and installed VKS documentation to inspect candidate release compatibility. A release being listed does not establish that it is a valid upgrade target for either cluster.
In DEV main.tf, change kubernetes_version
to the verified target identifier. Update cluster_class
only if required by the supported target combination. Leave worker count
and unrelated settings unchanged. The module maps these inputs to
spec.topology.version and
spec.topology.classRef.name.
cd terraform-create-vks-cluster/dev-cluster-01
terraform validate
terraform plan -out=upgrade.tfplan
terraform show -no-color upgrade.tfplanReview for an in-place update to the existing Cluster with the
intended release and any explicitly required ClusterClass change. A
generated local_file change is expected. Stop if the
Cluster is to be destroyed or replaced, its identity changes, or
unrelated topology changes appear. The Cluster resource staying in place
does not mean its nodes stay in place: VKS performs the node
rollout.
Apply only the reviewed plan:
terraform apply upgrade.tfplanVerify DEV with completion checks before changing QA. Then set the
verified target in QA main.tf, return to the repository root
(terraform-vks/), and repeat:
cd terraform-create-vks-cluster/qa-cluster-02
terraform validate
terraform plan -out=upgrade.tfplan
terraform show -no-color upgrade.tfplan
# Run only after reviewing the QA plan.
terraform apply upgrade.tfplanDo not use terraform init -upgrade to request a
Kubernetes upgrade; it changes Terraform provider/module dependency
selection, not kubernetes_version.
This change increases the existing single worker pool in each
cluster. Keep control_plane_count = 3, the Kubernetes
release, ClusterClass and VM/storage settings unchanged. In the
module "vks_cluster" block of DEV main.tf,
change the existing assignment:
worker_count = 5
The module maps this value to
spec.topology.workers.machineDeployments[0].replicas. Edit
the root’s literal assignment; worker_count is a module
input and is not declared as a root CLI variable, so
terraform -var='worker_count=5' is not the interface for
these roots.
| Capacity | Before, per cluster | After, per cluster | Increase across DEV and QA |
|---|---|---|---|
| Control-plane nodes | 3 | 3 | 0 |
| Worker nodes | 3 | 5 | 4 workers |
| Total nodes | 6 | 8 | 4 nodes; 16 total |
| Worker containerd volumes | 90 GiB | 150 GiB | 120 GiB; 300 GiB total |
These storage figures cover only the configured 30Gi
worker volumes. Add OS disks, application volumes and any temporary
rollout capacity when sizing quotas. Four additional workers also
require sufficient compute and network address capacity.
Plan DEV first:
cd terraform-create-vks-cluster/dev-cluster-01
terraform validate
terraform plan -out=scale-workers-5.tfplan
terraform show -no-color scale-workers-5.tfplanThe intended infrastructure change is an in-place Cluster update from three to five worker replicas, with no control-plane or release change. The local YAML may also be updated or replaced, including when its dated path changes. Do not accept a Cluster replacement. After reviewing the plan:
terraform apply scale-workers-5.tfplanWait for DEV to have five ready workers using the completion checks
below. Then change worker_count = 5 in QA
main.tf, return to the repository root
(terraform-vks/), and repeat:
cd terraform-create-vks-cluster/qa-cluster-02
terraform validate
terraform plan -out=scale-workers-5.tfplan
terraform show -no-color scale-workers-5.tfplan
# Run only after reviewing the QA plan.
terraform apply scale-workers-5.tfplanTerraform continues to manage one Cluster resource per root; it will
not show two new worker-VM resources in its plan. VKS reconciles the
replica change into new Machines. Keep Terraform as the lifecycle owner
rather than using a parallel kubectl scale or manifest-edit
workflow. Broadcom’s horizontal scaling
documentation
explains the underlying node-count operation.
After each apply, inspect the affected Cluster and its Machines. These read-only commands cover both targets:
kubectl config use-context supervisor
kubectl -n dev-namespace get clusters.cluster.x-k8s.io dev-cluster-01 -o yaml
kubectl -n dev-namespace get machines.cluster.x-k8s.io -l cluster.x-k8s.io/cluster-name=dev-cluster-01
kubectl -n dev-namespace get events --sort-by=.metadata.creationTimestamp
kubectl -n qa-namespace get clusters.cluster.x-k8s.io qa-cluster-02 -o yaml
kubectl -n qa-namespace get machines.cluster.x-k8s.io -l cluster.x-k8s.io/cluster-name=qa-cluster-02
kubectl -n qa-namespace get events --sort-by=.metadata.creationTimestampAuthenticate to each workload cluster using the established lab
access procedure. Run the following for each cluster, replacing
WORKLOAD_CONTEXT with its actual authenticated context
name; do not use supervisor for these workload checks:
kubectl config use-context WORKLOAD_CONTEXT
kubectl get nodes -o wide
kubectl get pods -ACompletion requires all of the following:
Available=True condition alone is insufficient.Finally, run a fresh terraform plan for the changed root
and review residual differences. Investigate unexpected Cluster drift. A
dated local YAML path can produce a file-only difference on a subsequent
day; do not confuse that with an incomplete Kubernetes rollout.
If apply times out or returns an inconsistent-result error, inspect the Cluster, Machines, events and Terraform state before issuing another mutation. The API may have accepted the request and controllers may still be working. Do not force conflicts, delete Machines, destroy/recreate the Cluster or reapply an old saved plan as an automatic recovery step.
Changing five workers back to three is a separate scale-down
operation involving workload placement and possible disruption; assess
PodDisruptionBudgets and capacity before planning it. Reverting
kubernetes_version is not a guaranteed rollback and can
request an unsupported downgrade. For a failed release rollout, use the
installed VKS recovery procedure and a supported forward fix or
application restore plan.
For a deliberate dependency upgrade, preserve current lock files and state, review the candidate providers’ release notes, and run the following for DEV first:
cd terraform-create-vks-cluster/dev-cluster-01
terraform init -upgrade
terraform validate
terraform planReview the lock-file changes and any resulting resource differences
before proceeding. After DEV verification, return to the repository root
(terraform-vks/), run
cd terraform-create-vks-cluster/qa-cluster-02, and repeat
the same plain Terraform commands. init -upgrade can update
both Kubernetes and Local providers within their constraints. It neither
changes the desired Kubernetes release nor proves that a cluster upgrade
has completed.
Only use import when the named Cluster already exists and is not
managed in the selected state. From the repository root
(terraform-vks/), run the applicable import below. The
second cd moves from DEV to QA if running both imports; for
QA alone, use cd terraform-create-vks-cluster/qa-cluster-02
instead:
# Existing DEV Cluster
cd terraform-create-vks-cluster/dev-cluster-01
terraform import module.vks_cluster.kubernetes_manifest.cluster 'apiVersion=cluster.x-k8s.io/v1beta2,kind=Cluster,namespace=dev-namespace,name=dev-cluster-01'
# Existing QA Cluster
cd ../qa-cluster-02
terraform import module.vks_cluster.kubernetes_manifest.cluster 'apiVersion=cluster.x-k8s.io/v1beta2,kind=Cluster,namespace=qa-namespace,name=qa-cluster-02'Generate and review a fresh plan for each imported Cluster before
applying. Resolve field ownership conflicts deliberately; the module
uses force_conflicts = false.
From the repository root (terraform-vks/), run only the
removal you intend. The second cd moves from DEV to QA if
removing both; for QA alone, use
cd terraform-create-vks-cluster/qa-cluster-02 instead:
# Delete DEV
cd terraform-create-vks-cluster/dev-cluster-01
terraform destroy
# Delete QA
cd ../qa-cluster-02
terraform destroyEach command displays its own plan and requires yes to
proceed. Review the namespace and Cluster name before confirming. Each
root manages its Cluster and generated YAML file; the existing namespace
and CA Secret remain.
After deletion, use the matching kubectl commands in Verify both clusters.
The Cluster should return NotFound, and its machine query
should return no resources. Run terraform state list from
the corresponding cluster directory as well. If deletion times out,
inspect remaining objects and deletion conditions before further action.
Preserve state until cleanup is resolved.
The Kubernetes provider uses the Supervisor endpoint and authentication from kubeconfig. This workflow does not call the vCenter management REST API or use shell provisioners.
| Resource | Provider-managed operations | Result |
|---|---|---|
module.vks_cluster.kubernetes_manifest.cluster |
Discovery/schema reads; GET/PATCH/DELETE under
/apis/cluster.x-k8s.io/v1beta2/namespaces/{namespace}/clusters/{name} |
Reads, applies or deletes the Cluster; provider handles transport/status checks, while VKS reconciliation requires separate verification |
local_file.cluster_manifest |
Local file creation/update/deletion | Writes the desired YAML; no API call or readiness check |
| Symptom | Check |
|---|---|
| Authentication or connection failure | Refresh Supervisor credentials and verify kubeconfig context, endpoint and TLS settings. |
| Missing Cluster kind during plan | Verify the selected Supervisor exposes the required CRD/version and allows discovery. |
| Admission rejection | Check release, ClusterClass, namespace bindings, quota, networks and referenced Secrets. |
| Timeout or inconsistent result after apply | Inspect the live Cluster and Terraform state before another mutation; the Cluster may already exist. |
| Unexpected replacement | Inspect state, taint and configuration differences before applying. |
The module includes the observed Supervisor default
podSecurityStandard.deactivated = false and marks topology
variables, version, labels and annotations as computed. These
accommodate admission defaults and release normalization. Earlier
configurations produced
Provider produced inconsistent result after apply despite
creating the Cluster. Use a fresh plan with the current module and
inspect state before recovery.
The following files record Terraform deployment activity and the resulting DEV and QA cluster inventory.
| Cluster | Output | Description |
|---|---|---|
dev-cluster-01 |
Terraform deployment output | Saved terminal transcript containing initialization, plans, applies,
deletion/recreation activity, Terraform output values, and cluster
checks. The final check records Available=True, three
available control-plane nodes, and three available workers. |
dev-cluster-01 |
Cluster documentation output | Read-only cluster inventory captured on 2026-10-22 at 13:21:53 GMT. Includes networking, node configuration and conditions, workloads, storage, CNI/CSI components, events, resource usage, and container images. Records six Ready nodes. |
qa-cluster-02 |
Terraform deployment output | Saved terminal transcript containing initialization, plans, applies,
deletion/recreation activity, Terraform output values, and cluster
checks. The final check records Available=True, three
available control-plane nodes, and three available workers. |
qa-cluster-02 |
Cluster documentation output | Read-only cluster inventory captured on 2026-10-22 at 13:22:26 GMT. Includes networking, node configuration and conditions, workloads, storage, CNI/CSI components, events, resource usage, and container images. Records six Ready nodes. |
The Terraform files are complete command transcripts, not captures of
terraform output alone. They retain intermediate
provisioning states and typed-command errors; follow the procedures in
this README when executing commands. The QA Terraform directory and
cluster are both named qa-cluster-02.
The cluster documentation files were generated by separate read-only collectors, not Terraform. Both report completed collection and zero writes attempted. Their embedded absolute paths refer to the original lab checkout; the links above open the copies in this repository.
These outputs describe historical observations. Use the cluster verification procedure to establish current readiness. See Generated files and outputs for the desired YAML manifests and Terraform output values.