Technical Architecture and Implementation Roadmap

Chapter Index

• Introduction — The Problem We Are Trying to Solve
• Objective
• Problem Statement Summary
• Executive Summary — Recommended Path
• Chapter 1: Existing State, Technical Documentation
• Chapter 2: Existing State, External and Global Traffic Management
• Chapter 3: VKS Adoption Journey, The Four Stage Roadmap
• Chapter 4: VCF Fleet Architecture, Multiple Sites Across Multiple Regions
• Chapter 5: Stage 2 Option 1, Brownfield Import of an Existing vCenter
• Chapter 6: Stage 3 Option 1, Brownfield vSphere Supervisor Single Zone
• Chapter 7: Stage 2 Option 2, Greenfield Workload Domains
• Chapter 8: Stage 3 Option 2, Greenfield vSphere Supervisor Three Zones
• Chapter 9: Stage 3 Load Balancer Option, Avi Load Balancer
• Chapter 10: Stage 4, F5 to Avi Load Balancer Migration
• Document Map — How the Chapters Fit Together
• Reference Table

Introduction — The Problem We Are Trying to Solve

This document defines the VCF 9 upgrade architecture for a strategic enterprise client, led by the client’s VCF 9 Upgrade Architect. The engagement has two parts: upgrading the client’s three existing, standalone vSphere 7/8 environments by importing each into VMware Cloud Foundation (VCF) 9.1 as a managed workload domain, and building new, dedicated workload domains to bring VMware vSphere Kubernetes Service (VKS) online and scale and standardize modern container workloads across the client’s infrastructure footprint. It establishes the target-state design, the constraints the design must satisfy, and the phased upgrade path required to reach production readiness.

Current State and Assumptions

• Infrastructure Footprint. The client operates three physically distinct, geographically distributed datacenters — West US, East US, and West Europe — running legacy, standalone vSphere 7 and vSphere 8 environments totaling approximately 45,000 physical cores.
• Network Fabric. A VMware NSX software-defined network overlay fabric is deployed, but only partially integrated across the two US sites; West Europe is not yet on the fabric.
• Application Delivery. External and global traffic management, multi-tenant application delivery, and edge load balancing are provided exclusively by enterprise F5 BIG-IP systems.

Objective

This document presents the VCF 9 upgrade path for the three standalone vSphere environments — each imported into VCF 9.1 as an upgraded workload domain — together with the new, VKS-dedicated workload domain design needed to bring VKS 3.7 into production. The design objective is the fastest, lowest-risk path to production readiness across all three physical locations.

Design Constraints

Constraint 1 — Minimal Workload Disruption. The upgrade methodology must maintain continuous infrastructure availability and policy alignment for existing production workloads running on the vSphere 8 environments. The design defines two tracks that run side by side — a brownfield upgrade of the existing environment and a greenfield build for VKS — that together eliminate risk to live traffic during the VCF 9.1 deployment.

Constraint 2 — Simplified Routing and Load Balancer Integration. The design must account for the pre-existing VMware NSX fabric and F5 BIG-IP load balancers as they integrate into the new VCF 9.1 and VKS 3.7 architecture, including network simplification options and alternative design choices (for example, vDS networking with Avi Load Balancer as a side-by-side or greenfield replacement). The design must provide resilient container-native ingress/egress routing, cloud-native microservices routing, and multi-site traffic management.

Constraint 3 — Standardized Automated Operations. The target architecture must adhere strictly to upstream Kubernetes standards and, at the later stages of the upgrade roadmap, fully leverage VCF SDDC Manager automated operations to manage the multi-cluster infrastructure lifecycle, ensuring repeatable deployment patterns for the rapid rollout of VKS 3.7.

Required Architectural Views

The design addresses three required architectural views, each mapped to a chapter in this document:

1. Global Multi-Site SDDC Topology — a high-level physical-to-virtual architectural view showing how the three physical datacenters are consolidated and managed under a federated, multi-site model. Addressed in Chapter 4.
2. Traffic Engineering and Packet Flow — the end-to-end routing path and policy enforcement points from the F5 BIG-IP edge, through the NSX software-defined overlay, into the VKS container-native ingress controllers, under a mandate for strict segmentation between namespaces. Addressed in Chapter 2, Chapter 6, Chapter 8, Chapter 9, and Chapter 10.
3. The Upgrade Pipeline Sequence — a phased implementation timeline defining the fastest operational path from vSphere 8 to a fully operational VKS environment. Addressed in Chapter 3 and the brownfield/greenfield chapters that follow it.

A note on release versions: the detailed design chapters in this document are built against the VCF 9.1 / vSphere Supervisor design library, which is the current VCF 9.x release line that the VKS 3.7 Kubernetes distribution is consumed through. Wherever this document references “VCF 9.1,” that is the platform release this VKS 3.7 rollout runs on — there is no separate VCF version to plan for.

Problem Statement Summary

Stripped of surrounding detail, this is the problem the rest of the document solves:

The chapters that follow document, in order: what the client operates today (Chapters 1–2), the overall roadmap and how its stages connect (Chapter 3), the multi-site fleet foundation every stage depends on (Chapter 4), the upgrade of the existing environment and the new, VKS-dedicated workload domain built alongside it — brownfield and greenfield — (Chapters 5–8), the Stage 3 load-balancer choice (Chapter 9), and the Stage 4 completion of the traffic-management modernization that retires F5 (Chapter 10).

Executive Summary — Recommended Path

This engagement has two workstreams, run side by side rather than as competing alternatives: upgrading the client’s existing vSphere 8 environment into VCF 9.1 by importing it as-is, and building a new, dedicated workload domain to bring VKS 3.7 into production. Because all three of the client’s sites already run vSphere with NSX, import — not migration or rebuild — is the correct mechanism for the upgrade workstream everywhere it applies.

1. Stand up a new VCF 9.1 Fleet spanning all three sites, following the multi-site fleet design in Chapter 4, with US WEST hosting the fleet-wide services (VCF Operations, VCF Automation, License Server, SDDC Manager).
2. Upgrade workstream — import, not migrate, each site’s existing vCenter and NSX environment into the fleet as a workload domain, per Chapter 5 — production VMs are never moved or rebuilt, only brought under VCF management. Where a site’s imported cluster has spare capacity, Supervisor can also be activated on it directly, using the Single Management Zone with Combined Workload Zones model, per Chapter 6.
3. VKS workstream — build a new, dedicated workload domain using the Multi-Rack Layer 2 vSphere Cluster Model and the Centralized Connectivity Model, per Chapter 7, and activate the Three Management Zones Supervisor model on it, per Chapter 8 — giving VKS its own purpose-built capacity and zone-level resilience, independent of the imported estate.
4. Choose a Stage 3 load-balancer for Supervisor on each workload domain — the built-in NSX Load Balancer, or Avi Load Balancer where Layer 7, WAF, GSLB, or analytics are required — per Chapter 9.
5. Complete the private cloud operating model at Stage 4 by extending Avi fleet-wide and retiring F5 through the seven-phase, wave-based migration in Chapter 10.

The two workstreams do not block each other. The imported workload domain modernizes the existing vSphere 8 estate’s management plane without touching a single running VM, and can optionally take on some VKS capacity of its own if a site has room for it. The new, VKS-dedicated workload domain (Chapter 7 and Chapter 8) is built independently, on its own timeline, so VKS gets purpose-built capacity and zone-level resilience rather than being retrofitted onto infrastructure sized for something else.

Chapter 1: Existing State, Technical Documentation

Client: Strategic enterprise, 3 datacenters (West US, East US, West Europe)

Scope of this document: What the client has today, and what it takes for that hardware to qualify for VMware Cloud Foundation (VCF) 9.1. No future-state design is covered here.

1. Why vSphere support dates matter

The client’s three datacenters run a mix of vSphere 7 and vSphere 8. Both versions are still usable today, but they are on different support clocks — and vSphere 7 is close to running out.

General Support is the period where Broadcom actively patches, fixes bugs, and answers support tickets. Once this ends, the client is running unsupported software for production workloads. Technical Guidance is a reduced tier after General Support ends — existing documentation and best-effort guidance only, no new patches or bug fixes.

vSphere 7 already exited General Support on October 2, 2025. Any hosts still running vSphere 7 are outside the normal support window today. This is the most urgent driver for the upgrade. vSphere 8 has runway until October 2027. It is not an emergency, but it is also not permanent — it does not eliminate the need to move to VCF 9.1 / ESXi 9.1, it just means vSphere 8 hosts have more breathing room than vSphere 7 hosts.

The support clock, not just the desire to adopt Kubernetes, is what makes this an active modernization project rather than a “someday” initiative.

2. Sizing the existing environment: two conservative estimates

The client’s infrastructure footprint is stated as ~45,000 physical cores spread across three datacenters, on Dell hardware, with a 50/50 split between rack and blade/modular servers. Because the client did not supply exact server counts, two conservative, Intel-based estimates are used to translate “45,000 cores” into a concrete, physical picture: how many servers, what models, how many racks. Both estimates land on the same ~45,000 cores — they simply assume different-generation Dell hardware, which changes the server count and rack footprint.

2.1 Estimate A — 2021 Intel-generation hardware (higher server count)

This estimate assumes the client’s fleet was purchased around 2021, using 3rd Generation Intel Xeon Scalable CPUs — a conservative (lower cores-per-server) assumption, which produces a higher server and rack count.

Assumptions - 80 physical cores per server (2 × Intel Xeon 3rd Gen, up to 40 cores each) - 50/50 split between rack and blade/modular servers - Evenly distributed across the three datacenters

Hardware used in this estimate

Totals

Per-datacenter split

Rack layout per datacenter

Rack servers (shown for the largest site; other sites are the same pattern at 94 servers):

Blade/modular servers (MX750c sleds, 8 sleds per MX7000 chassis, max 2 chassis per rack):

Rack count per datacenter (West Europe shown — 187 servers, one fewer blade sled than West US/East US):

West US and East US follow the same layout with 94 rack servers and 94 blade sleds each, also landing at 11 racks per site.

If the client’s hardware is from the 2021 era, it takes 563 servers across roughly 33 racks total (11 per site) to deliver the ~45,000 cores they operate today.

2.2 Estimate B — 2023 Intel-generation hardware (lower server count)

This estimate assumes a refresh cycle to 4th Generation Intel Xeon Scalable CPUs — more cores per server, so the same ~45,000 cores are delivered by fewer, denser servers.

Assumptions - 112 physical cores per server (2 × Intel Xeon 4th Gen, up to 56 cores each) - 50/50 split between rack and blade/modular servers - Evenly distributed across the three datacenters (67 rack + 67 blade per site)

Hardware used in this estimate

Totals

Per-datacenter split

Rack layout per datacenter (identical for all three sites under this estimate — 67 rack + 67 blade servers each)

Rack servers:

Blade/modular servers (MX760c sleds, 8 sleds per MX7000 chassis, max 2 chassis per rack):

Rack count per datacenter:

This layout is the same at West US, East US, and West Europe.

If the client’s hardware is from the 2023 era, the same ~45,000 cores fit in 402 servers across 27 racks total (9 per site) — 161 fewer servers and 6 fewer racks total than Estimate A, purely because newer CPUs pack more cores per socket.

2.3 Why two estimates instead of one

The client stated a core count but not a hardware generation or exact server count. Rather than guess, both a lower bound and upper bound are provided using real, currently-supported Dell server families:

Both estimates are deliberately conservative — they use mainstream, well-documented Dell rack/blade models rather than the newest or highest-density options, so the resulting server and rack counts represent a realistic worst case, not a best case. The true environment likely sits somewhere in this range (a mixed fleet), which is why both models are carried forward together rather than collapsed into a single number. Whichever end of the range the real environment sits closer to, the hardware in both estimates is confirmed compatible with VCF 9.1 (see Section 3 below), so the range does not block planning.

3. Hardware qualification for VCF 9.1

Moving to VCF 9.1 is not just a software upgrade — the underlying hardware (server, storage controller, network adapter, firmware) must be certified for ESXi 9.1, and the storage must be either a certified vSAN ReadyNode configuration or a supported external storage array.

The qualification chain:

VCF 9.1 support
      =  ESXi 9.1 support
      +  supported storage controller / NIC / firmware
      +  vSAN ReadyNode configuration
         OR supported external storage array

All checks are made against the Broadcom Compatibility Guide (the authoritative source for VCF/vSphere hardware compatibility): https://compatibilityguide.broadcom.com/search?program=server&persona=live&column=partnerName&order=asc

3.1 Estimate A hardware (2021-era: R750 / MX750c) — confirmed supported

Dell’s own ESXi 9.x compatibility matrix confirms the R750 and MX750c, with Intel Xeon SP 83xx/63xx/53xx/43xx CPUs, support both ESXi 9.0 and ESXi 9.1. Source: https://www.dell.com/support/manuals/en-lk/vmware-esxi-9-x/vmware_9.x_compatibility_matrix_pub/dell-yx5x-poweredge-systems

3.2 Estimate B hardware (2023-era: R760 / MX760c) — confirmed supported

Dell’s matrix confirms the R760, R660, MX760c, and C6620, with Intel Xeon SP 64xx/54xx/44xx/34xx CPUs (and Xeon SP 94xx on R760/R660/C6620), support both ESXi 9.0 and ESXi 9.1. Source: https://www.dell.com/support/manuals/en-us/vmware-esxi-9-x/vmware_9.x_compatibility_matrix_pub/intel-sapphire-rapids-processor-and-raptor-lake-processor

3.3 What this means for planning

• Both conservative estimates use hardware that is confirmed on the VCF 9.1 / ESXi 9.1 compatibility list. There is no hardware-generation blocker to moving forward — whether the real environment turns out to be closer to the 2021-era or 2023-era model, the servers qualify.
• Server compatibility is only one leg of the qualification chain. Before any workload domain is built, the specific storage controller, NIC, and firmware revisions in use at each site still need to be checked individually against the Broadcom Compatibility Guide, and the storage configuration (vSAN ReadyNode or supported external array) must be confirmed per cluster.
• This document does not assume vSAN vs. external storage for the existing environment — that determination requires an inventory of the actual storage arrays/controllers per site, which is a follow-on discovery step, not a hardware-generation question.

Summary

Chapter 2: Existing State, External and Global Traffic Management

Client: Strategic enterprise, 3 datacenters (West US, East US, West Europe)

Scope of this document: How user traffic reaches applications today. This describes the current state only. No future-state design is covered here.

1. The short version

Today, all application delivery — deciding which datacenter serves a user, and then load balancing inside that datacenter — is handled entirely by F5 BIG-IP. There is no VCF and no VKS in this picture yet. The systems receiving that traffic are plain vSphere 7/8 virtual machines, running in three separate, standalone datacenters: West US, East US, and West Europe.

There are two F5 components in play, and they do two different jobs:

2. How a request flows today

The path a request takes is the same three-hop pattern at every site: a global DNS decision, a local load-balancing decision, then the backend VM. Each datacenter box in the diagram also carries its rack footprint from the two conservative sizing estimates, so the traffic-management view and the physical-capacity view tie back to the same three sites:

Step by step, matching the numbered hops in the diagram:

1. DNS Query. A user’s device resolves the application’s DNS name. That request lands on F5 BIG-IP DNS/GTM, not a plain DNS server — GTM is authoritative for the application’s public name.
2. Best Site Selected. BIG-IP DNS/GTM continuously monitors the health and performance of all three datacenters — is the site up, how loaded is it, how far away is the user — and returns the IP address of the best datacenter for that user right now. This is the global decision: which site handles the request. The datacenters themselves take no part in this decision; it is made entirely at the GTM tier.
3. VIP / Local LB. The user’s traffic then arrives at that site’s F5 BIG-IP LTM. LTM owns the local decision — it terminates the connection at a virtual IP (VIP) for the application, checks the health of individual backend VMs, applies SSL/TLS termination, enforces session persistence, and load-balances the request across the correct pool of VMs using Layer 4/Layer 7 policy.
4. Delivery. The request finally reaches the target application, running as a virtual machine on vSphere 7 or vSphere 8 inside that datacenter’s rack footprint (11 racks under the 2021 estimate, 9 racks under the 2023 estimate — see the rack-elevation diagrams in the technical documentation).

The key distinction to remember: BIG-IP DNS/GTM makes the global “which site” decision once, up front. BIG-IP LTM makes the local “which server” decision, on every request, inside the chosen site. Together they form the entire application delivery control plane today — there is no other load balancing or ingress layer in front of these VM workloads, and the vSphere layer has no awareness of the other two sites.

3. What sits behind F5 today

Behind the F5 layer, the three datacenters are not yet unified:

• Each site is a standalone vSphere 7/8 environment — there is no VMware Cloud Foundation and no shared management layer across sites yet.
• Each site’s virtual machines are self-contained; there is no container platform (no VKS, no Kubernetes) in this state.
• F5 is the only thing providing multi-site awareness. The vSphere layer itself does not know or coordinate across West US, East US, and West Europe — that coordination happens entirely in the BIG-IP DNS/GTM tier.

4. Why this matters for the modernization project

This existing state is the baseline every future design has to work with and around:

• F5 is the incumbent control plane. Any future architecture needs to explicitly decide what F5 keeps doing (typically: the global/edge tier) versus what a new load-balancing layer takes over (typically: in-cluster, container-native routing).
• There is no application-delivery integration with Kubernetes today, because there is no Kubernetes today. Introducing VKS means introducing a new ingress/egress layer that has to either sit behind F5, integrate with F5, or replace part of F5’s role — this is a design decision for the next phase, not something the current state answers.
• The three sites are operationally independent. Global traffic steering is a BIG-IP DNS/GTM configuration, not an infrastructure-level federation. This is a separate concern from the vSphere-layer standalone state described in the technical documentation.

Summary

Chapter 3: VKS Adoption Journey, The Four Stage Roadmap

Client: Strategic enterprise, 3 datacenters (West US, East US, West Europe)

Scope of this document: How the client moves from its current, standalone vSphere environment toward VMware Cloud Foundation (VCF) and VMware vSphere Kubernetes Service (VKS), one stage at a time. This is the map of the whole project: every stage below now links to the detailed solution document that was built for it, so this document is the index as much as it is the narrative.

1. The journey

The client does not need to adopt the entire VMware Cloud Foundation and VKS stack in one move. The journey happens in four progressive stages, each one adding capability on top of the last, without requiring a rebuild of what came before. A workload domain can sit at any stage for as long as the organization needs — there is no forced timeline to reach Stage 4.

Stages 2 and 3 each have two tracks that run side by side — an upgrade track, brownfield, that imports and modernizes what’s already running, and a VKS track, greenfield, that builds a new, dedicated workload domain for Kubernetes — and this project has a detailed design document for both tracks at both stages, plus everything needed for Stage 4.

2. The four stages at a glance

The key architectural message: each stage is a superset of the one before it. Moving from Stage 1 to Stage 2 does not touch the running VMs — it adds a management layer on top of the existing vCenter and ESX hosts. This is why the client can start modernizing now without putting live production workloads at risk, and why each stage below has a document the client can hand to an implementation team on its own.

3. Stage 1 — Traditional Environment (where the client is today)

This is the client’s current state, fully described in Existing State — Technical Documentation and Existing State — External and Global Traffic Management.

What’s running: - vCenter, managing traditional virtualization operations - ESX hosts underneath it, running VM workloads - F5 BIG-IP DNS/GTM and LTM, handling all global and local traffic management in front of those VMs

What’s not present yet: - No VMware Cloud Foundation - No centralized, multi-site operations layer - No Kubernetes services — infrastructure operations are entirely VM-centric - No software-defined load balancing — F5 is hardware-appliance-based

In plain terms: this is standalone vSphere, administered site by site, the same way the client runs West US, East US, and West Europe today. It is a normal, valid operating model — it’s just the starting point, not the destination.

4. Stage 2 — Platform Foundation Modernization

This is the first real step of the journey, and it is deliberately low-risk: it adds a management and operations layer on top of the existing vCenter and ESX hosts, without replacing them.

What gets introduced: - Upgrade to vSphere 9.1 — brings the hosts onto currently-supported, VCF 9.1-qualified software - VCF Operations — centralized visibility across the environment (monitoring, capacity, health) - License Server — centralized license management, instead of managing licenses per site - Enhanced password and certificate management — tightening operational security practices as part of the same modernization pass - A VCF Fleet spanning all three sites — West US, East US, and West Europe become three VCF Instances under one fleet, detailed in VCF Fleet Architecture

What does not change at this stage: - The same vCenter continues to manage the same ESX hosts - No Kubernetes services are enabled yet — this stage is entirely about the operational foundation, not about running containers - Existing VM workloads are not disrupted; VCF Operations and License Server sit alongside the existing management plane, they do not replace it - F5 keeps doing its job unchanged — Stage 2 does not touch traffic management

Why this stage matters: Stage 2 is the foundation every later stage depends on. Centralized operations visibility and licensing have to exist before it makes sense to roll out Kubernetes (Stage 3) or a full private cloud operating model (Stage 4) across three sites — otherwise each site would be repeating the same manual setup independently.

Two tracks into Stage 2

Every site brings its own existing vSphere + NSX environment, and this project runs two tracks side by side rather than choosing one over the other — a full solution document exists for each:

Because every one of the client’s three sites already runs vSphere with NSX, import (not migration or rebuild) is how the existing environment gets upgraded — it’s the only supported way to bring an existing site into VCF 9.1 without abandoning what’s already there. The greenfield track runs alongside it: a new workload domain, built specifically to host VKS, rather than retrofitted onto the imported estate.

Multi-site reference design: for a client running three datacenters, Stage 2 is where the multi-site operating model gets established, using VMware’s reference blueprint VCF Fleet with Multiple Sites Across Multiple Regions — see VCF Fleet Architecture for the full design, including which VCF Instance hosts the fleet-wide services (VCF Operations, VCF Automation, License Server, SDDC Manager) versus which instances are lighter, instance-level peers.

5. Stage 3 — VKS Adoption

With the Stage 2 foundation in place — the imported workload domain and the new, VKS-dedicated workload domain built alongside it — Stage 3 turns on vSphere Supervisor, which is what actually makes VKS Kubernetes clusters possible. Kubernetes workloads run in the new, purpose-built workload domain, alongside the existing VMs that continue running, untouched, in their upgraded workload domain.

What gets introduced: - vSphere Supervisor, activated on top of the workload domain’s vSphere Zone(s) - VKS Clusters and vSphere Pods, consumed through vSphere Namespaces - A networking stack (NSX Segment Networking or NSX VPC Networking, depending on path) that gives every namespace its own Kubernetes API endpoint, ingress, and load balancer - A load balancer choice for that networking stack — see below

What does not change at this stage: - Existing VM workloads keep running unmodified, in namespaces of their own if needed - The workload domain’s vCenter, NSX, and storage from Stage 2 are reused, not rebuilt - Global traffic management (F5, still) is untouched — Stage 3 is about what happens inside a workload domain, not how users reach it from outside

Two Supervisor models, one per workload domain

Stage 3’s Supervisor deployment model depends on which workload domain it’s being activated on — the imported one, or the new VKS-dedicated one — since each has a different number of vSphere Zones to work with:

The practical difference: the imported workload domain’s single-zone model gives Supervisor host- and cluster-level resilience (via vSphere HA/DRS on the one zone it has); the new workload domain’s three-zone model gives Supervisor zone-level resilience — its three control plane VMs are spread one per zone, so losing an entire zone still leaves Supervisor with quorum. Both are valid designs for their respective workload domain — the choice follows directly from which workload domain Supervisor is being activated on, not a separate decision.

Networking detail matters here too: the imported workload domain uses NSX Segment Networking (a shared Tier-0 Gateway with an auto-created Tier-1 Gateway and load balancer per namespace); the new, VKS-dedicated workload domain uses NSX VPC Networking (one VPC per namespace, inside an NSX Project, aligned with VCF Automation’s project model). Both documents include the concrete IP/CIDR sizing requirements needed to actually build the networking, not just the conceptual model.

The load balancer choice: NSX Load Balancer or Avi Load Balancer

Document: vSphere Supervisor Load Balancer Option: Avi Load Balancer

Whichever networking stack a site ends up on, Supervisor needs a load balancer underneath it, and both networking stacks support the same two choices:

This is a load-balancer decision, not a zone-model or connectivity decision — it doesn’t change which workload domain (the imported one, or the new VKS-dedicated one) Supervisor is being activated on, and it doesn’t have to be decided at the same time as that choice. The NSX Load Balancer is the simpler built-in default. Avi Load Balancer is the richer choice — the same Controller/Service-Engine architecture used everywhere else in this project — and is worth choosing where a site needs Layer 7 load balancing, WAF, GSLB, or deep traffic analytics beyond what the built-in NSX Load Balancer provides. Document 09 covers the Avi option in full: the 1:1 Avi-Controller-to-NSX-Manager deployment model, the Centralized Transit Gateway data path for Supervisor traffic, and how Avi Controllers and Service Engines place onto the three-region VCF Fleet from document 04.

6. Stage 4 — Full Private Cloud Operating Model

This is the final stage: the client’s environment stops being “a workload domain that happens to run Kubernetes” and becomes a genuine private cloud operating model — self-service provisioning, policy-driven automation, fleet-wide lifecycle management, and, critically, a modern application-delivery layer that finishes replacing what F5 does today.

What gets introduced: - NSX, SDDC Manager, VCF Automation, and VCF Management Services — the full VCF control plane, fleet-wide (already touched on in the Stage 2 fleet architecture, fully active by this stage) - Avi Load Balancer extended beyond Supervisor to the client’s entire application estate, retiring F5 completely

Avi as the F5 replacement, fleet-wide

Document: F5 Replacement: Migrating to VMware Avi Load Balancer

This closes the loop all the way back to Stage 1: the F5 BIG-IP DNS/GTM and LTM estate described in the existing-state traffic management document is retired and replaced by the same Avi platform already available as a Stage 3 load-balancer option (document 09) — Avi GSLB takes over the global “which site” decision from F5 BIG-IP DNS/GTM, and Avi Virtual Services on Avi Service Engines take over the local “which server” decision from F5 BIG-IP LTM. Every F5 concept (pools, VIPs, health monitors, SSL profiles, iRules, partitions) has a direct Avi equivalent, mapped out in full in document 10.

The migration itself is deliberately not a cutover: F5 and Avi run in parallel through a seven-phase migration (discover F5 → build the Avi foundation → recreate local LTM services as Avi Virtual Services → recreate global GTM services as Avi GSLB → pilot one low-risk application → cut over production application by application → decommission F5 in waves). Nothing about this migration is blocked by, or blocks, any other stage — it can run on its own timeline once the Stage 4 fleet-wide control plane exists.

Why this is genuinely the last stage: Stage 4 isn’t complete with just NSX/SDDC Manager/VCF Automation turned on — a private cloud operating model still needs an answer for the application traffic that predates this whole project. Any site that chose Avi as its Stage 3 Supervisor load balancer (document 09) is already running Avi; this stage extends that same platform to carry everything F5 carries today, so the client ends up on one load-balancing platform — Avi — across VM, VCF, and VKS workloads, instead of running F5 and Avi side by side indefinitely. A site that chose the NSX Load Balancer for Supervisor in Stage 3 can still adopt Avi fleet-wide here — the two decisions are independent.

What does not change at this stage: - Workload domains, Supervisor, and VKS clusters built in Stages 2 and 3 are not rebuilt — Stage 4 adds automation, fleet-wide operations, and load balancing on top of them - The migration to Avi is per-application and reversible mid-flight (a rollback path back to F5 is kept open until an application is proven stable on Avi) — nothing is a forced, one-time cutover

7. How the stages connect, end to end

Reading the whole journey as one line, per site:

Stage 1 (standalone vCenter/ESX/F5) → Stage 2 (VCF Fleet established; the existing environment is upgraded via import, and a new, dedicated workload domain is built for VKS via greenfield) → Stage 3 (vSphere Supervisor activated on the new workload domain — and, where wanted, on the imported one too; VKS clusters start running behind the NSX Load Balancer or Avi Load Balancer) → Stage 4 (NSX/SDDC Manager/VCF Automation complete the private cloud operating model; Avi Load Balancer is extended fleet-wide to retire the F5 estate).

Because Stage 2 and Stage 3 each run an upgrade track and a VKS track side by side, and because the three sites don’t have to move through the journey in lockstep, the client can run West US, East US, and West Europe at different stages simultaneously — for example, one site already on Stage 4 while another is still validating Stage 3 — without any stage depending on the others being at the same point.

Summary

Chapter 4: VCF Fleet Architecture, Multiple Sites Across Multiple Regions

Client: Strategic enterprise, 3 datacenters (West US, East US, West Europe)

Scope of this document: The actual solution design for running VMware Cloud Foundation (VCF) across multiple, geographically separated sites as a single managed fleet. This is the Stage 2 (Platform Foundation Modernization) architecture referenced in the VKS Adoption Journey document.

Source blueprint reviewed for this design: VCF Fleet with Multiple Sites Across Multiple Regions.

1. The solution

Instead of building three separate, disconnected VCF deployments — one per datacenter — the client builds one VCF Fleet: three VCF Instances, one per site, that operate independently day-to-day but are tied together under shared fleet-level management, shared identity, and shared operations visibility. The diagram maps this directly onto the client’s three datacenters: Region A — US WEST, Region B — US EAST, and Region C — EUROPE WEST.

2. The core idea: one “first” instance, everyone else is a peer

This is the single most important design decision in the whole blueprint, and it is easy to miss on first read:

• The first VCF Instance runs two sets of components: the components that manage its own site (instance-level), plus the components that manage the entire fleet (fleet-level) — VCF Operations Cluster, VCF Operations for Networks Cluster, VCF Automation Cluster, License Server, SDDC Manager, and the Cloud Proxy.
• Every additional VCF Instance runs only its own instance-level components — its own Management vCenter, Workload vCenter, NSX Manager Clusters, and management services — and reports up to the fleet-level services running in the first instance.

In the diagram, this is why VCF Instance 1 (Region A — US WEST) has a full 4-column grid of components in its Management Domain Cluster, while VCF Instance 2 (Region B — US EAST) and VCF Instance 3 (Region C — EUROPE WEST) each have a lighter 2-column grid — neither duplicates fleet management, licensing, or automation. Both consume those services from Instance 1 over the network.

Site assignment for this design: US WEST is designated the “first” VCF Instance and hosts the fleet-wide services; US EAST and EUROPE WEST run as instance-level peers. US WEST was selected here as the fleet anchor as a working assumption (typically the site with the strongest network position and operational team); this can be revisited during detailed site planning if a different site is better positioned to hold the fleet role.

3. Walking through the diagram

Top bars (span all three sites): - Self-Service with VCF Automation — the self-service consumption layer. Any tenant, in any region, requests infrastructure through the same VCF Automation front end. - VCF Operations — the single-pane-of-glass monitoring and visibility layer across every site in the fleet.

Inside each VCF Instance — Management Domain Cluster: - Management vCenter / Workload vCenter — one vCenter manages the physical hosts, a separate vCenter manages workload placement. This separation is standard VCF practice, repeated identically at every site. - Management NSX Manager Cluster / Workload NSX Manager Cluster — the same split applied to networking: one NSX Manager for infrastructure networking, one for workload networking. - VCF management services — the baseline services every instance needs to operate (present at all three sites). - Fleet-only components (Instance 1 / US WEST in the diagram): VCF Operations Cluster, VCF Operations for Networks Cluster, VCF Automation Cluster, License Server, SDDC Manager, Virtual Network Appliance. - Instance-level component present at all three sites: Cloud Proxy — a lightweight local collector that feeds telemetry back to the fleet-level VCF Operations Cluster in US WEST.

Inside each VCF Instance — Workload Domain Cluster: - NSX Edge (×2) — provides north-south routing and load-balancing capacity for that site’s workloads. - Supervisor VM Cluster — the vSphere Supervisor control plane. This is the component that turns the workload domain into a Kubernetes platform (the mechanism behind VKS in Stage 3).

Below the instances — storage and network fabric: - Storage Site A1 / Storage Site B2 / Storage Site C3 — each site’s own primary storage. Optional asynchronous replication is shown between US WEST ↔︎ US EAST and US EAST ↔︎ EUROPE WEST. This is not synchronous, stretched storage — each site keeps its own storage, and data is optionally copied between adjacent sites for resilience, not shared in real time. Replication does not have to be limited to this chain — any site pair can replicate directly if the client’s disaster-recovery plan calls for it. - Virtual Networking — the NSX-backed overlay running on top of the physical fabric, local to each site. - Physical Network — the underlying physical fabric, connected between sites over standard Layer 3 routing (US WEST ↔︎ US EAST ↔︎ EUROPE WEST). There is no requirement for a stretched Layer 2 network between regions — ordinary routed IP connectivity between sites is sufficient, including across the US-to-Europe hop.

4. Key design decisions (from the reference blueprint)

The blueprint’s design profile makes a specific set of choices for this topology. Restated in plain English:

5. What this means for the client’s three sites

Mapped onto West US, East US, and West Europe:

• Each site becomes one VCF Instance, built the same way — same Management Domain Cluster pattern, same Workload Domain Cluster pattern. A VCF Instance is not limited to a single workload domain: a site can run both the imported workload domain from Chapter 5 and the new, VKS-dedicated workload domain from Chapter 7 side by side, each with its own Workload Domain Cluster reporting up to the same instance-level and fleet-level services shown here.
• US WEST additionally hosts the fleet-level services in this design.
• All three sites are connected over ordinary Layer 3 routed networking — no exotic network requirement to stand this up, including the transatlantic hop to EUROPE WEST.
• Storage stays local to each site, with optional asynchronous replication shown chained US WEST ↔︎ US EAST ↔︎ EUROPE WEST — this is a choice per site pair, not an all-or-nothing requirement, and any two sites can replicate directly if needed.
• This is exactly the platform foundation Stage 2 of the VKS Adoption Journey calls for: once this fleet is in place, Stage 3 (enabling VKS on the Supervisor Control Plane already shown in each Workload Domain Cluster) is an incremental step, not a redesign.

6. Operations monitoring: Cloud Proxy and VCF Operations across three regions

The Cloud Proxy component appears in every VCF Instance in the diagram above (Section 3), but it’s worth explaining on its own because it’s easy to confuse with a networking or traffic component. It is neither.

What Cloud Proxy actually does

Cloud Proxy is the local operations collector and integration bridge for a VCF Instance. It lets the central VCF Operations platform pull metrics, logs, events, tasks, alarms, and health data from the local components at that site — vCenter, ESXi, NSX, vSAN, SDDC Manager, and VCF Management Services — without those components having to talk to the central platform directly over the WAN.

A few things Cloud Proxy is not: - It does not carry application traffic. - It does not replace NSX, SDDC Manager, or F5 BIG-IP. - It is not a second copy of VCF Operations — it is a lightweight local collector that feeds the one central platform.

In a multi-site design, a Cloud Proxy is deployed close to the VCF Instance it monitors — one per region, at minimum — so that operational data collection stays local while visibility stays centralized.

The monitoring model

VCF Operations is deployed once, centrally, in the fleet’s primary management region — US WEST in this design, matching the same site that hosts the other fleet-wide services described in Section 2. It is the single console for dashboards, alerts, capacity planning, log analysis, compliance, lifecycle visibility, inventory, health, and issue tracking across the whole fleet. Broadcom documents VCF Operations as mandatory in VCF 9.x — every fleet has exactly one.

Every region runs its own Cloud Proxy, including US WEST itself. Each Cloud Proxy talks only to its own local VCF Instance and forwards what it collects up to the central VCF Operations platform. West US, East US, and West Europe each get one; a second Cloud Proxy per region is a reasonable addition for local high availability, but a second full VCF Operations cluster is not — East US and West Europe do not need their own VCF Operations deployment unless one of them requires a genuinely independent operations control plane (a separate fleet, data-sovereignty requirement, or mandated regional isolation).

Recommended monitoring design, by layer

What each region actually hosts

Neither East US nor West Europe gets its own VCF Operations cluster or its own License Server in this design — both are fleet-level services that live once, in US WEST, and are consumed by every region over the network, the same pattern already established for VCF Automation and SDDC Manager in Section 2.

7. Disaster recovery: asynchronous replication across three regions

Section 3 mentioned that storage replication between sites is optional and asynchronous. This section explains what that actually means and how disaster recovery (DR) is implemented on top of it.

What “asynchronous” actually means

Asynchronous replication copies selected VM workloads from one region to another after the write has already been committed at the source site. It is not zero data loss — that’s what synchronous replication provides, and synchronous replication requires very low, metro-distance latency that doesn’t exist between West US, East US, and West Europe. Instead, asynchronous replication gives the client a defined RPO (Recovery Point Objective) — a chosen, known amount of possible data loss, such as 5 minutes, 15 minutes, 1 hour, or 4 hours, depending on how the replication schedule is configured.

The write itself is never held up waiting for the remote site — production keeps running at full local speed, and the replica catches up shortly after.

Why the client wants this

Asynchronous replication in this design exists for disaster recovery and workload mobility, not for everyday local high availability (local HA is already handled inside each site by vSphere HA and the cluster design covered in Section 4).

Broadcom describes the VCF DR model as a primary site and recovery site architecture: production workloads are replicated to a secondary site so they can resume operations if the primary site becomes unavailable. Recovery sites do not need to be physically identical to the primary — they only need enough resources to run the protected workloads, not a mirror-image build.

The recommended VCF 9.1 building blocks

The clean VCF 9.1 implementation combines:

• VMware Live Recovery / Site Recovery — the DR orchestration engine, working primarily between the vCenter inventories of the source and target sites
• Enhanced vSphere Replication — the underlying replication mechanism
• VCF Operations visibility — fleet-wide DR status, protection state, and alerting
• Optional vSAN Protection and Recovery — for vSAN-native protection policies
• Optional array-based replication — where the client’s storage array already provides its own replication engine

Protection topology

Each VCF Instance runs its own VLR / Site Recovery locally — DR is not a separate fourth platform, it’s a capability layered onto the same three VCF Instances already described in this document, with status rolling up into the same central VCF Operations console used for day-to-day monitoring (Section 6).

Two ways to structure who protects whom:

• Option A — Regional Pair Model (recommended default): each site has its own dedicated recovery partner — West US protects East US, East US protects West US, and West Europe protects East US. This is what the diagram above shows.
• Option B — Shared Recovery Site Model (alternative): every site replicates into one shared recovery site instead of pairing 1:1. This lowers recovery-site cost, but that one site becomes the recovery target for everyone, concentrating risk and recovery capacity in a single place.

Recommendation: pair sites by application criticality and business ownership, not by blindly replicating everything everywhere — not every workload needs cross-region DR, and treating all of them identically adds cost without adding protection where it matters.

Example protection pairing:

Summary

Chapter 5: Stage 2 Option 1, Brownfield Import of an Existing vCenter

Client: Strategic enterprise, 3 datacenters (West US, East US, West Europe)

Scope of this document: One clear message — because every client site already runs vSphere with NSX, the only supported way to upgrade West US, East US, and West Europe onto VMware Cloud Foundation (VCF) 9.1 is to deploy a new VCF 9 fleet and import each existing vSphere + NSX environment into it as a workload domain. This document explains that upgrade track in full: what it means, what it requires, how it works, and exactly what happens to the existing NSX network. This is Stage 2 of the modernization journey. The VKS track — building a new, dedicated workload domain to bring VKS into production — is covered separately and is out of scope here.

Reference: Import an Existing vCenter to Create a Workload Domain (Broadcom techdocs).

The mechanism this upgrade relies on

Import an Existing vCenter to Create a VCF 9.1 Workload Domain. VCF 9.1 can bring an already-running vCenter under VCF management directly. The existing vCenter, its clusters, hosts, storage, and NSX Manager all come in as-is — nothing is copied, moved, or rebuilt. This is the mechanism the entire upgrade track relies on.

Deploy New VCF 9 Environment / Import Workload Domain. The general pattern: stand up a brand-new VCF 9 fleet on unused hardware, then import an existing environment into it afterward.

How this plays out for the client’s environment

This is the same three-phase pattern shown above, expanded with every NSX-specific check, precheck, and validation step that matters for an environment that already has NSX deployed — the level of detail an implementation team actually needs.

What “import” actually means

This is not a migration tool — nothing is copied, moved, or rebuilt. Broadcom’s own framing is simple: you import the existing vCenter, clusters, hosts, storage, and NSX into VCF management. The VMs stay exactly where they are, and the existing NSX Manager — with its Tier-0/Tier-1 gateways, segments, and firewall policy — comes in as-is.

Before: - Standalone vSphere 8 environment (if a site is still on vSphere 7, it must be upgraded to a supported vSphere 8 build first — vSphere 7 cannot be imported directly) - vCenter 8.x, ESXi 8.x clusters, VM workloads - vSAN / FC / NFS / iSCSI storage - Existing NSX Manager (4.2.x), registered to vCenter, with Tier-0/Tier-1 gateways and segments already configured

After import: - A VCF 9.1 Instance now exists, with its Management Domain (SDDC Manager, VCF Operations, VCF Management Services) - The existing environment becomes an Imported VI Workload Domain inside that instance — same vCenter, same ESXi clusters, same VM workloads, same storage, same NSX Manager with the same T0/T1 gateways and segments, now visible and managed through VCF

One important constraint: the import is all-or-nothing at the vCenter level. Every cluster inside that vCenter comes in together — there is no cluster-by-cluster import. If some clusters should end up in different workload domains, or if the vCenter contains unsupported or legacy clusters, that has to be resolved (split, reorganized, or remediated) before the import runs, not after.

Requirements

Prerequisites (must be true before you start)

• Any site still running vSphere 7 must first be upgraded to a supported vSphere 8 build (8.0 Update 3a or later) — vSphere 7 does not meet the minimum version for import and cannot be imported directly into VCF 9.1. This upgrade has to happen before the import workflow, as its own separate step.
• SSH is enabled on the existing vCenter instance
• All ESX hosts use FQDNs, not short names, in the vSphere inventory
• The existing vSphere infrastructure matches the supported configurations listed in “Supported configurations” below

Minimum component versions

The vCenter and ESX minimums both start at 8.0 — there is no path that imports a vSphere 7 environment directly. A site still on vSphere 7 has to be upgraded to a supported vSphere 8 build first, as a standalone step, before the import prerequisites above can be satisfied.

One version trap worth flagging explicitly: NSX 9.1 does not support vCenter 8.0 Update 3a. If the plan is to bring the existing NSX Manager onto a shared NSX 9.1 instance (or upgrade it to a 9.1 NSX instance as part of this workstream), the vCenter must first be upgraded to 9.1 — importing at 8.0 U3a and expecting NSX 9.1 compatibility does not work. Since the client’s environment already runs NSX, this version pairing has to be checked and resolved before import, not discovered during it.

Additional NSX-specific checks and requirements

Because the client’s environment already has NSX deployed, these checks matter more than the general “no NSX yet” path documented by Broadcom — validate all of them before scheduling an import:

• NSX registration state. Confirm the vCenter is currently registered with NSX as expected. With an existing registration, the import registers vCenter as a Compute Manager in NSX; it does not create a new NSX Manager. Clusters already NSX-prepared stay prepared; clusters that are not NSX-prepared today stay unprepared after import — the import does not retroactively prepare them.

• NSX Manager health. All NSX Manager cluster nodes should be healthy and reachable before import — an unhealthy NSX Manager is a precheck failure waiting to happen.

• Single NSX Manager per vCenter. VCF does not support multiple NSX Manager instances managing one vCenter. If the environment somehow has more than one, this has to be resolved first.

• Network segment placement. The NSX Manager must be on the same network segment as the vCenter appliance being imported. NSX Manager and vCenter sitting on different network segments is explicitly unsupported.

• NSX / vCenter version compatibility matrix. If this NSX Manager is (or will be) shared across workload domains, the version pairing determines what feature set is available:

  NSX version	vCenter version	Resulting state
  NSX 9.1	vCenter 9.1	Full features
  NSX 9.1	vCenter 9.0.x	Pinned state — features limited to what vCenter 9.0 supports
  NSX 9.0.x	vCenter 9.0.x	Full features
  NSX 4.2.x	vCenter 8.0.x	Legacy mode

• NSX Federation. If the existing NSX deployment is part of an NSX Federation (a Global Manager spanning multiple sites), only the Local Manager is imported. The Global Manager relationship — and anything Federation-managed, such as T0 gateways stretched across sites — is not automatically re-established and needs to be validated separately after import. For a 3-site client like this one, confirm up front whether Federation is in use anywhere before assuming a clean import.

• Edge cluster / Edge node inventory. If NSX is 9.0 or later and Edge clusters exist, the import automatically discovers and pulls the Edge node VMs into VCF inventory — and resets their credential passwords as part of that process. Plan to retrieve the updated passwords from the VCF credential store afterward, and note this as an operational step, not a side effect to be surprised by.

• IPv4-only shared NSX constraint. If the NSX instance being reused is deployed in an IPv4-only VCF domain, the new workload domain must communicate over IPv4 only between its components, even if the workload domain itself is otherwise configured for dual-stack (IPv4/IPv6) networking.

• Distributed Firewall (DFW) impact. The import automatically activates NSX on every Distributed Virtual Port Group (DVPG) it finds, which in turn automatically activates DFW on all of them — with default-allow rules for Layer 2 and Layer 3 traffic. This is a firewall posture change on day one of the import, not a neutral action; see “Post-import lifecycle” below for how to handle it.

Supported configurations

These are the configuration boundaries the existing environment has to fit inside before import. Anything outside this list needs to be remediated first.

Storage — supported: - Enough free space for a full VCF deployment - A datastore shared across, accessible from, and writable by every host in the cluster - Any supported vSphere storage type - vSAN Stretched Clusters (minimum 3 ESX hosts per availability zone, plus a witness host) - Two-node vSAN clusters for ROBO deployments (2 hosts + 1 witness) - vSAN OSA clusters, provided deduplication/compression settings match across hosts

Storage — not supported: vSAN OSA clusters using compression-only configurations.

Network — supported: - vSphere Distributed Switch (VDS) 8.0 or later - Statically assigned VMkernel IPs (NSX Host TEPs are the one exception — DHCP is fine there) - A dedicated network for vSphere vMotion - VDS with LACP enabled - Clusters sharing a VDS - DNS with both forward and reverse records - A shared NSX Manager across multiple vCenter instances (as long as Enhanced Linked Mode isn’t in use) - NSX Bare Metal and VM Edge nodes

Network — not supported: Cisco virtual switches, a vCenter without a VDS, custom distributed port groups, non-default vCenter ports, dynamically allocated VMkernel IPs, or multiple NSX Managers managing a single vCenter.

Compute — supported: - The vCenter VM hosted on the default cluster in the management domain - Clusters using vSphere Configuration Profiles - Clusters using vSphere Lifecycle Manager images - Clusters using fully automated vSphere DRS - Standalone ESX hosts or single-host clusters, as long as at least one other compliant cluster also exists

Compute — not supported: vCenter with Enhanced Linked Mode, an NSX Manager already shared across different VCF instances, baseline-based lifecycle management, manual or partial DRS, partial cluster imports, vCenter HA (VCHA) clusters, or a vCenter that’s already connected to SDDC Manager.

What this means for the client: before importing any of the three datacenters, each site’s actual storage/network/compute configuration needs to be checked against this list. A site running Cisco virtual switching, VCHA, or partial DRS, for example, would need remediation before it qualifies — this is a concrete discovery task, not a formality.

The import workflow

Walking through the diagram’s middle phase:

1. In VCF Operations, go to Operate → Inventory → Detailed View, select the target VCF Instance, and choose “Import a vCenter”.
2. Enter a name for the new workload domain.
3. Select the vCenter to import, plus its existing, already-registered NSX Manager.
4. Confirm the certificate thumbprints.
5. The vCenter registers as a Compute Manager in NSX. Clusters already NSX-prepared stay prepared; clusters that weren’t prepared before import remain unprepared.
6. If Edge clusters exist on that NSX Manager (NSX 9.0+), the Edge node VMs are auto-discovered and pulled into VCF inventory — their credential passwords are reset as part of this step.
7. Run the built-in prechecks — this is the gate that catches unsupported configurations before anything changes.
8. Review the settings and click Finish.

A few behaviors worth knowing about going in: existing IPv4/IPv6 dual-stack networking carries over as-is (subject to the IPv4-only constraint noted in “Additional NSX-specific checks and requirements” above if the NSX instance is shared into an IPv4-only domain). And the import workflow automatically activates NSX on the Distributed Virtual Port Groups it finds, which in turn automatically turns on the Distributed Firewall (DFW) — with default-allow rules — across all DVPGs in the imported cluster. That’s a security-relevant side effect to plan for, not just a technical detail; it’s addressed directly in “Post-import lifecycle” below.

Post-import lifecycle: aligning to the VCF 9.1 BOM

Being imported does not mean being upgraded. Right after import, the workload domain is under VCF management but is still running whatever vCenter/ESXi/NSX versions it had going in. Getting it onto the VCF 9.1 Bill of Materials (BOM) is a separate, subsequent lifecycle step, shown as the third phase in the diagram:

1. Imported vSphere 8 workload domain — VMs untouched, now visible and managed through VCF
2. Review DFW activation on all DVPGs in the imported cluster — the import turns DFW on with default-allow rules for Layer 2 and Layer 3 traffic. This is not a hardened state; treat it as a checklist item on day one, not an assumption of “secure by default.” Two options: tighten the DFW rules to the client’s actual policy immediately, or — if DFW activation isn’t wanted yet — deactivate NSX on the affected DVPGs via the NSX UI or the Transport Node Collection API, and revisit later.
3. Validate existing T0 / T1 gateways and segments — confirm the existing NSX logical topology came through unchanged (see “What happens to the existing NSX T0, T1 gateways, and segments” below for what to expect and how to validate this).
4. Apply configuration updates — align local configuration to VCF-managed expectations
5. Download / map VCF 9.1 binaries — pull the correct binaries for the target BOM
6. Run VCF lifecycle prechecks — the gate before any upgrade is applied
7. Upgrade NSX / vCenter / ESXi as required — brought up to the versions the VCF 9.1 BOM specifies
8. Workload domain aligned to VCF 9.1 BOM — the end state, fully lifecycle-managed by SDDC Manager going forward

This two-step separation (import first, upgrade second) is what keeps the brownfield path low-risk: the import itself is a management-plane change, not a software upgrade, so it doesn’t carry upgrade risk. The upgrade to the VCF 9.1 BOM happens afterward, on its own schedule, with its own prechecks gate.

What happens to the existing NSX T0, T1 gateways, and segments

This is the question every network team asks first, and it deserves a direct answer: the import does not touch NSX logical network topology.

Why: the vCenter import operates at the compute manager registration level — it links (or confirms the link between) vCenter and its existing NSX Manager, and it determines which clusters are NSX-prepared. It does not read, rebuild, or recreate Tier-0 gateways, Tier-1 gateways, segments, groups, or DFW policies. Those are configuration objects that live inside the existing NSX Manager, and that NSX Manager is brought into VCF’s managed inventory as a whole, unchanged — not disassembled and reconstructed. Broadcom’s own description of the “existing NSX registration” path backs this up directly: clusters that are not already NSX-prepared are explicitly not reconfigured during import — the import only acts on what’s eligible, and leaves everything else alone.

What that means concretely:

• Existing Tier-0 gateways (north-south routing, BGP/static routing to the physical network) are retained exactly as configured. They keep running before, during, and after the import.
• Existing Tier-1 gateways (per-tenant or per-application routing) are retained exactly as configured, with the same segment attachments they had before.
• Existing segments (the NSX-backed logical networks VMs are attached to) are retained — VM connectivity to those segments does not change as a result of the import itself.
• The one thing that does change automatically is DFW activation state on DVPGs (see “Post-import lifecycle” above, item 2) — that’s a firewall policy action, not a T0/T1/segment topology change, but it’s real and needs to be reviewed immediately.
• If NSX Federation is in play, only the Local Manager comes into VCF (see “Additional NSX-specific checks and requirements” above). Any T0 gateway that’s stretched across sites via Federation needs its Global Manager relationship validated separately — the import does not know about, or preserve, Federation-level constructs beyond the Local Manager.

Recommended validation step: Broadcom’s documentation does not explicitly state “T0/T1 and segments are preserved unchanged” in those exact words — that behavior is inferred from how the import is scoped (compute-manager registration, not network reconfiguration). Because of that, don’t just assume it — export or snapshot the existing NSX Manager’s Tier-0/Tier-1 gateway configuration, segment list, and DFW rules before the import, and diff that against the same configuration after the import completes, before signing off on the import as done. This turns an inference into a verified fact for this specific environment.

Where this fits for the client

Post-import, two operational limitations to plan around: hosts cannot be added to an imported cluster without going through the vSphere Client (not through VCF Operations directly), and password management via the VCF Operations Console isn’t available for imported domains out of the box (Broadcom documents a workaround under KB 388859).

Summary

Chapter 6: Stage 3 Option 1, Brownfield vSphere Supervisor Single Zone

Client: Strategic enterprise, 3 datacenters (West US, East US, West Europe)

Scope of this document: The Supervisor design for the imported, upgrade-track workload domain — Stage 3 of the modernization journey, for sites that want extra Kubernetes capacity on the imported estate itself, in addition to the new, VKS-dedicated workload domain covered separately. This covers the simplest supported Supervisor deployment model: one vSphere Zone, its storage, and its network connectivity. The Supervisor design for the new, VKS-dedicated workload domain (multiple vSphere Zones across multiple clusters) is a separate document and out of scope here.

References: Single Management Zone with Combined Workload Zones Model and NSX Segment Connectivity Model Overview (Broadcom techdocs), plus “vSphere Supervisor with NSX Segment Networking: Architecture” (VCF Solution Architecture and Design training material, page 2-35).

The solution in one paragraph

Once the imported workload domain exists (via the import path covered in the previous document), adding Kubernetes capacity to it is a matter of turning on vSphere Supervisor on top of it. The simplest way to do that is the Single Management Zone with Combined Workload Zones Model: one vSphere Zone, mapped to one vSphere Cluster, does double duty as both the Supervisor’s management zone and its workload zone. No second cluster, no extra zone, no extra complexity — just the existing (imported) cluster, activated for Supervisor.

Design decisions at a glance

Four decisions define this model, and all four are the simplest available option:

The zone model itself

Reading the diagram top to bottom, this is what “turning on Supervisor” actually builds:

• Workload Domain — the outer boundary. This is the same workload domain created by the import (or by a fresh deployment), containing its own vCenter and NSX.
• vSphere Zone 1 (Management and Workload) — the one zone this model uses. It maps to a single vSphere Cluster, and that cluster hosts everything: Supervisor’s own management components and the application workloads that consume Kubernetes.
• vSphere Supervisor — the control layer that turns the cluster into a Kubernetes platform. It runs three Control Plane VMs for resilience (not one — a single control plane VM would be a single point of failure).
• vSphere Namespaces — the unit of isolation inside Supervisor. Each namespace gets its own resource limits, permissions, and storage. The diagram shows two examples side by side:
  • Namespace A running a VKS Cluster — a full guest Kubernetes cluster, for teams that want a real Kubernetes API.
  • Namespace B running plain Virtual Machines — for workloads that don’t need Kubernetes at all, provisioned through the same self-service model.
• Datastore 1 — the storage backing the whole zone (detailed next).

Why this is the right starting point for the client: every one of the three imported workload domains (West US, East US, West Europe) already has exactly one vSphere Cluster per site in the simplest case. This model activates Supervisor directly on that cluster — no re-architecture, no second cluster to build first.

Key design considerations that shaped this model (per Broadcom’s design guidance): vSphere Supervisor cluster topology, vSphere Supervisor zone design, and control plane VM placement and sizing (Small / Medium / Large — sized to the expected number of namespaces and API load, not fixed by this model).

A hard requirement, not optional: the vSphere Zone selected for Supervisor must have vSphere HA enabled and vSphere DRS running in Fully Automated or Partially Automated mode, and it cannot already be assigned to another Supervisor instance. Both the vSphere Zone name and the Supervisor name must also follow RFC 1123 naming rules (lowercase, alphanumeric, hyphens only, starting with a letter or number) — worth checking early, since a bad name is a rejected activation, not a warning.

Storage: the zonal datastore

A zonal datastore is simply a datastore that’s scoped to one vSphere Zone:

• It’s attached to every host in that zone’s underlying vSphere Cluster.
• It’s only accessible to workloads running inside that specific zone — not shared with any other zone.
• High availability comes from the vSphere Cluster itself (via vSphere HA), not from the datastore spanning multiple zones.

In plain terms: storage doesn’t need to be redesigned for this model. Whatever datastore already backs the imported cluster becomes the zonal datastore for Supervisor, as long as it meets the standard requirements (shared across, accessible from, and writable by every host — the same storage rule already covered in the import requirements).

Networking: vSphere Supervisor with NSX Segment Networking

This is the actual wiring diagram for how Supervisor’s networking works once NSX Segment Networking is in use — not just the abstract Tier-0/Tier-1 model, but where the load balancers, control plane VMs, pods, and ESX hosts each sit, and how they all reach the management plane.

The two halves of the diagram:

• vSphere Namespaces (left) — this is where tenant workloads live. Supervisor’s NCP (NSX Container Plugin) automatically builds a dedicated Tier-1 Gateway and Load Balancer for the namespace, connected up to the shared Tier-0 Gateway. Inside, a VKS Cluster gets its own Control Plane VMs on their own logical segment, and its Kubernetes pods run as vSphere Pods behind the load balancer.
• Supervisor System Namespaces (right) — this is Supervisor’s own control infrastructure: its Control Plane VMs, the vSphere Pods that run Supervisor’s system services, and the underlying ESX hosts, each on their own logical segment, fronted by their own Tier-1 Gateway and load balancer.

What the load balancer actually exposes, and why it’s different on each side: - On the vSphere Namespaces side, the load balancer publishes three things: the apiserver virtual IPs for the VKS Kubernetes Cluster (so users can reach that cluster’s Kubernetes API), the Kubernetes Load Balancer Service (for type: LoadBalancer services created inside that cluster), and Ingress Virtual IPs (for ingress traffic into that cluster). - On the Supervisor System Namespaces side, the load balancer only needs two: the apiserver virtual IPs for the Supervisor cluster itself, and Ingress virtual IPs for Supervisor-level ingress. There’s no separate Kubernetes Load Balancer Service entry here, because this side isn’t running tenant workloads — it’s running Supervisor itself.

The functional behaviors that matter operationally (from the architecture reference):

1. NCP creates one shared Tier-1 Gateway for system namespaces, and a separate Tier-1 Gateway plus load balancer for each namespace, by default. Every Tier-1 Gateway connects up to the same Tier-0 Gateway and down to a default segment — this is automatic, not something to hand-configure per namespace.
2. Workloads inside the same namespace share one SNAT IP for north-south traffic. From the outside, all outbound traffic from a namespace’s workloads looks like it’s coming from one address.
3. East-west traffic between namespaces happens without SNAT. Namespace-to-namespace connectivity is preserved at the real source IP — SNAT is a north-south (outbound-to-the-world) behavior only, not something that gets in the way of workloads talking to each other.
4. Each VKS cluster gets its own separate network segment inside its Tier-1 Gateway, specifically for isolation — one VKS cluster’s pod network is not the same broadcast domain as another’s.
5. The Tier-0 Gateway is associated with the NSX Edge cluster, which is what actually provides the routing path out to the external network via the uplink VLAN.

Tying it back to the management plane: both halves ultimately depend on the same NSX Manager and vCenter, which sit on the Management Network — the same management plane that governs the rest of the imported workload domain. Nothing about enabling Supervisor networking introduces a second, separate management path.

Network Requirements for Supervisor with NSX Segment Networking

The diagram above shows the shape of NSX Segment Networking; this section gives the concrete numbers — how many IPs, which CIDR sizes, which components — needed to actually build it.

1. Management Network Requirements

2. Workload Network Requirements

3. NSX Network Requirements

Notes on TEPs and VLANs: - ESXi hosts and NSX Edge nodes both act as tunnel endpoints — a TEP IP is assigned to each host and each Edge node. - The ESXi host VTEP and the Edge VTEP must both have an MTU size greater than 1600. - Because ESXi host TEP IPs form an overlay tunnel with the Edge nodes’ TEP IPs, the VLAN IPs on both sides must be routable to each other. - A separate, additional VLAN is required to provide North-South connectivity to the Tier-0 Gateway. - IP pools can be shared across clusters, but the host overlay IP pool/VLAN must not be shared with the Edge overlay IP pool/VLAN — unless host TEP and Edge TEP traffic use different physical NICs, in which case they can share the same VLAN.

NSX Federation — a note for a 3-site client

Because the client operates three datacenters, NSX Federation is directly relevant, not a theoretical feature:

• Federation gives one point of management for workload networking and security policy across all three sites’ NSX instances.
• It extends Layer 2 and Layer 3 topology across sites, and centrally manages security groups and policy.
• Each workload domain can still keep an independent control plane, which is what makes Federation a good fit for disaster-recovery designs — one site’s NSX control plane failing doesn’t take down another site’s.
• Important constraint: once network topology is extended across workload domains via Federation, NSX multi-tenancy (Projects) can no longer be used for that topology — all configuration has to be done centrally by the provider admin in NSX’s default Project, not per-tenant.
• Federation’s Global Manager cluster only needs Layer 3 connectivity between sites — no Layer 2 stretch required, which matches the client’s existing routed, non-stretched network design (see the existing-state traffic management documentation).
• Cross-site traffic needs an MTU above 1280 bytes at minimum (1700+ recommended for 1500-byte workload VMs, 9000 for jumbo frames) — worth checking against the client’s current inter-site network MTU before enabling Federation.

What this means for the client’s three sites

• Each of West US, East US, and West Europe activates Supervisor the same way: one vSphere Zone per site, mapped to that site’s imported cluster, using a zonal datastore and NSX Segment Networking on the NSX Manager already imported with it.
• No new clusters need to be built to reach this stage — this model deliberately reuses what’s already there.
• If the client later wants centrally managed, cross-site security policy (or is planning site-to-site DR for Kubernetes workloads), NSX Federation layers on top of this same model without requiring a redesign — the Tier-0/Tier-1 structure stays the same, only the Global Manager relationship is added.
• Activation can happen per site, independently — either during that site’s workload domain creation, or afterward from vCenter — so the three sites do not need to move through this stage in lockstep.

Summary

Chapter 7: Stage 2 Option 2, Greenfield Workload Domains

Client: Strategic enterprise, 3 datacenters (West US, East US, West Europe)

Scope of this document: The VKS track — standing up a brand-new, dedicated VCF workload domain to bring VKS into production — Stage 2 of the modernization journey, run alongside the brownfield upgrade track covered separately. This covers the two design building blocks that define this workload domain: the Multi-Rack Layer 2 vSphere Cluster Model (compute/fault-domain design) and the Centralized Connectivity Model with Shared Tier-0 Gateway Per Tenant (networking design). Stage 3 (enabling VKS on this workload domain) is a separate, later step and is out of scope here.

References: Multi-Rack Layer 2 vSphere Cluster Model and Centralized Connectivity Model (Broadcom techdocs).

The solution in one paragraph

Greenfield is the opposite of the brownfield import path covered elsewhere in this project: instead of bringing an existing vSphere + NSX environment under VCF management as-is, a brand-new workload domain is built from scratch on new or repurposed hardware. That means every design decision — compute layout, fault tolerance, network connectivity — gets made fresh, rather than inherited from whatever was already running. The two decisions that matter most for a new workload domain are covered here: how the compute cluster is laid out across racks for resilience, and how tenants get network connectivity to the outside world.

Compute: Multi-Rack Layer 2 vSphere Cluster Model

The problem this solves: a single-rack cluster protects against a host failure, but not against losing an entire rack (a failed top-of-rack switch, a power incident, a maintenance mistake). This model protects against exactly that, by spreading vSphere cluster hosts across multiple racks.

How it works, reading the diagram bottom to top: - Hosts sit physically in four separate racks (Rack 1 through Rack 4), two hosts per rack in this example. - All of this sits inside one Workload Domain — this is not three separate workload domains, it’s a single workload domain built with three internal clusters. - The diagram shows three Layer 2 vSphere Clusters, and each cluster is a vSphere Zone — Cluster 1 is Zone 1, Cluster 2 is Zone 2, Cluster 3 is Zone 3. Each is its own pool of capacity, independently managed by its own vSphere HA and DRS, and every one of the three is stretched across the same four physical racks. - Every infrastructure network a given cluster/zone needs — VM Management, ESX Management, vMotion, IP Storage (NFS/vSAN), NSX Host TEP — is stretched across all four racks as a single Layer 2 broadcast domain for that zone. A host in Rack 3 is on the exact same VM Management network as a host in Rack 1 within its own zone, not a routed, separate one. - Above that, each rack’s switches carry all three zones’ stretched Layer 2 traffic between racks, all within one set of Layer 2 Adjacencies.

Scenario: dedicated racks per zone (six racks)

The diagram above stretches all three zones across the same four racks. That’s one valid physical layout — it isn’t the only one. Here’s a second, equally valid scenario, not an alternative option to choose instead of the first: six racks are available, and each zone is confined to its own dedicated pair of racks rather than spreading across all of them.

Multi-Rack vSphere Cluster and Zone Model

One VCF Workload Domain contains three vSphere clusters. Each vSphere cluster is mapped to one vSphere Zone. The ESXi hosts for each cluster are distributed across multiple racks to provide rack-level resiliency within the zone.

The vSphere Zone is a logical availability construct associated with a vSphere cluster — it is not a rack-level object. Physical racks provide the underlying host placement and failure-domain design; the zone is the logical boundary layered on top of that physical placement, and the two don’t have to line up the same way every time.

Workload Domain
  |
  +-- vSphere Zone 1 / Cluster 1
  |     +-- Hosts distributed across Rack 1, Rack 2
  |
  +-- vSphere Zone 2 / Cluster 2
  |     +-- Hosts distributed across Rack 3, Rack 4
  |
  +-- vSphere Zone 3 / Cluster 3
        +-- Hosts distributed across Rack 5, Rack 6

How this differs from the first diagram: in the four-rack layout, all three zones share the same four racks — every zone’s hosts are physically interleaved with the other two zones’ hosts. In this six-rack scenario, each zone gets its own pair of racks, with no physical overlap between zones at all. Both give each zone the same logical protection (loss of one rack doesn’t take out the whole zone, because a zone’s hosts sit in two racks) — the difference is purely how the physical racks are assigned, not how the zones behave logically.

When dedicated racks per zone make more sense than shared racks: when the client wants a zone’s failure domain to be physically, not just logically, independent of the others — for example, if Zone 1’s racks are on a different power feed, a different network uplink pair, or even planned for a different physical location within the datacenter than Zones 2 and 3. Sharing racks (the first diagram) is more space-efficient; dedicating racks (this scenario) is more strictly isolated.

Why three zones: preparing for the Three Management Zones Supervisor model

This three-cluster layout isn’t compute design for its own sake — it’s building the exact physical foundation Stage 3’s greenfield Supervisor design needs: the Three Management Zones with Combined Workload Zones Model.

• One vSphere Zone per cluster. In the simplest mapping (used here), each vSphere Zone maps to exactly one vSphere Cluster. VCF’s zone model also allows a zone to aggregate multiple clusters for extra scale, but three single-cluster zones is the straightforward starting point.
• “Combined” means each zone carries both roles. Every one of the three zones is used for both Supervisor management and application workloads — there’s no separate, dedicated management-only cluster sitting apart from the workload clusters, unlike a traditional management-domain/workload-domain split.
• This is what protects Supervisor from a cluster outage. Because Supervisor’s management components are spread across all three zones rather than concentrated in one place, losing an entire cluster/zone doesn’t take Supervisor itself down — the surviving two zones keep the control plane available. That protection is the entire point of building three zones instead of one.
• No extra clusters needed for application workloads. Because each zone is already combined (management + workload), application teams don’t need a fourth cluster built specially for them — they land in the same three zones Supervisor’s own components run in.

Requirements per zone (per cluster):

Total requirements, across all three zones:

• Minimum host count: 3 zones × 3 hosts minimum = 9 ESX hosts minimum, spread across the same 4 racks — a starting baseline; real sizing grows with application workload demand on top of it.
• Fifteen stretched Layer 2 networks in total (5 network types × 3 zones), all riding the same physical rack switches — this is the real reason rack switch capacity and VLAN planning matter more here than in a single-zone design.
• Three independent points of failure tolerance: each zone tolerates one host failure on its own, and the workload domain as a whole tolerates losing one entire zone without Supervisor going down.
• Storage note, looking ahead to Stage 3: this compute layout is what makes a cross-zone datastore possible later — a single logical datastore spanning all three zones, giving infrastructure-level protection against a whole-zone failure (as opposed to the single-zone model’s zonal datastore, which only protects at the cluster level within one zone).
• Networking note, looking ahead to Stage 3: this three-zone workload domain is exactly what the Centralized Connectivity Model (covered next) is designed to connect — one shared Tier-0 Gateway serving the whole workload domain, regardless of which zone a tenant’s workloads land in.

Why this is simpler than it sounds: because every network is one broadcast domain instead of one-per-rack, this model needs fewer VLANs and subnets than a Layer 3 multi-rack design would. The tradeoff is that Layer 2 has to actually be stretchable across those racks in the client’s physical network — this model assumes that capability exists, it doesn’t create it.

Resilience, concretely: protection operates at two levels — vSphere HA protects against a single host failing, and the multi-rack spread protects against an entire rack failing. A cluster sized to tolerate one host failure keeps running even if a whole rack goes dark, because the surviving hosts are physically in other racks.

Sizing, at minimum:

These are the general-purpose VCF cluster minimums from the design guide, for reference — they apply when a cluster is dedicated to one role only (a traditional, separate management domain, for example):

In the three-zone Combined Workload Zones model used here, that traditional split doesn’t apply — each zone already carries both roles, which is why the per-zone minimum above is 3 hosts (the same floor as a simple management cluster), not a separate, larger number for a dedicated management cluster plus more for workload clusters on top of it.

Requirements worth calling out explicitly: - One set of clusters per VCF workload domain — this workload domain’s three zones are not shared with any other workload domain. - vSphere Lifecycle Manager images (not baselines) — this keeps firmware and vendor add-ons manageable as one unit. - vSphere HA enabled for every cluster, full stop. - Every ESX host configured as an NSX transport node, under a single overlay transport zone.

Practical recommendations from the design guide: use separate vSphere Distributed Switches to keep storage, NSX, and vMotion traffic apart (minimum 2 NICs per switch); prefer vSAN ESA as the primary storage engine; assign NSX Host TEP IPs from static pools rather than DHCP; and enable vSphere DRS in fully automated mode so load balances across racks without excessive vMotion churn.

Networking: Centralized Connectivity Model with Shared Tier-0 Gateway Per Tenant

The problem this solves: every tenant needs a path to the outside world, but building a dedicated Tier-0 Gateway (and its NSX Edge nodes) per tenant is expensive and often unnecessary. This model lets multiple tenants share one Tier-0 Gateway and its Edge node resources, while still keeping each tenant’s networking logically separate.

What is a “tenant,” concretely?

“Tenant” is a deliberately generic word in NSX’s design language — an NSX Project — and it can map to whatever organizational boundary actually needs network isolation. It doesn’t have to mean a different company or a different customer. For an enterprise client running its own private cloud, a tenant is most often an internal organizational boundary:

• A department or cost center. The client’s HR organization and Finance organization are a textbook example: two tenants, each with its own VPC(s), its own NAT and IP addressing, and — critically — no visibility into each other’s subnets by default. HR’s payroll systems and Finance’s general ledger systems sit on genuinely separate networks, even though they share the same physical Tier-0 Gateway underneath.
• An application environment. Production, staging, and development can each be modeled as a tenant, so a misconfiguration or a compromised host in staging has no network path to production.
• A business unit or subsidiary. In a larger enterprise, separate business units often need independent IP address ranges, independent NAT, and independent security policy — exactly what a dedicated C-TGW and VPC set provides.
• An external customer, for a client that also happens to host or resell infrastructure to others — the same isolation mechanism works whether the boundary is internal or external.

The one property every tenant shares, no matter what it represents organizationally: it gets its own Centralized Transit Gateway, its own VPC(s), and its own IP address space, and traffic never crosses from one tenant to another except through whatever explicit routing/firewall policy is deliberately configured — never by default.

How it works, reading the diagram top to bottom: - External Connectivity and BGP bring routes in from the physical network to a single, shared Tier-0 Gateway, running in Active/Active mode. - Each Tenant (an NSX Project) gets its own Centralized Transit Gateway (C-TGW), connected to that shared Tier-0. The C-TGW runs in Active/Standby mode and is where tenant-level NAT and VPN services attach. - Inside each tenant, one or more VPCs connect to that tenant’s C-TGW, each with its own VPC Gateway, its own NAT, and its own Subnet carrying the actual workload VMs. - This is the same building block VCF Automation’s tenancy models are built on, and it also supports vSphere Supervisor deployments directly — it’s not exclusive to one consumption model.

Why share Tier-0 instead of dedicating one per tenant: it’s simply more resource-efficient — fewer NSX Edge nodes to deploy and operate, while tenants still get logical isolation through their own C-TGW and VPCs. The tradeoff is that all inter-VPC and north-south traffic for every tenant ultimately passes through the same shared Edge nodes, so there’s a real (if manageable) risk of resource contention between tenants under heavy load — worth monitoring, not a reason to avoid the model outright.

When to choose the shared model vs. a dedicated Tier-0 per tenant:

Requirements worth calling out explicitly: - A minimum of two NSX Edge nodes in the Edge cluster, for high availability. - An enterprise-routable external IP range (a /24 or larger is preferred) for NAT, public subnets, and load balancer VIPs. - A non-routable IP range (/23 or larger preferred) assigned to the private-TGW block, for inter-VPC connectivity within a tenant. - For vSphere Supervisor / VCF Automation specifically: N-S Services and Default Outbound NAT (Auto-SNAT) both need to be enabled on the Default VPC Connectivity Profile.

Practical recommendations from the design guide: run Tier-0 in Active/Active to scale out to eight NSX Edge nodes; use a unique private BGP ASN per Tier-0 to avoid loop-detection issues; peer each Edge node with two separate physical devices for redundancy; and for Supervisor’s load-balancing needs specifically, use Large/X-Large Edge appliances or bare-metal NSX Edge rather than the smaller form factors.

Where this fits for the client

• This is the path to use when a new workload domain is being built — a new site, a new cluster on new hardware, or a deliberate clean rebuild — rather than bringing an existing vSphere+NSX environment under management (that’s the brownfield import path covered separately in this project).
• The Multi-Rack Layer 2 Cluster Model is directly applicable to the client’s rack-based footprint documented earlier in this project (11 racks per site under the 2021 estimate, 9 under the 2023 estimate) — spreading each of the three zones across several of those racks is exactly the fault-domain protection this model is designed for.
• Building the workload domain as three zones from the start is what sets the client up for the Three Management Zones Supervisor model in Stage 3 without any rework later — the compute layout doesn’t need to change when VKS gets enabled, only Supervisor activation gets layered on top.
• The Centralized Connectivity Model fits naturally with a multi-tenant design: if the client’s three sites host multiple departments (HR, Finance, and so on), business units, or application environments as separate tenants, this model gives each one logical isolation without the cost of a dedicated Tier-0 Gateway per tenant per site — and it’s the same connectivity model this three-zone workload domain is designed to use.
• Both models can be adopted independently — a client could use the Multi-Rack Layer 2 Cluster Model with a dedicated Tier-0 instead, or the Centralized Connectivity Model on a single-zone workload domain. They’re presented together here because this is the combination Stage 2 Option 2 uses as its greenfield baseline.

Summary

Chapter 8: Stage 3 Option 2, Greenfield vSphere Supervisor Three Zones

Client: Strategic enterprise, 3 datacenters (West US, East US, West Europe)

Scope of this document: The greenfield vSphere Supervisor design for enabling VKS — Stage 3, Option 2 of the modernization journey. This is the greenfield counterpart to the brownfield single-zone design covered earlier: instead of one combined zone, three vSphere Zones are activated across the three-cluster Workload Domain already built in the previous document. This covers the zone model itself, its cross-zone storage, its networking, and how it all wires together end to end for VKS consumption.

References: Three Management Zones with Combined Workload Zones Model and Centralized Connectivity Model (Broadcom techdocs).

The solution in one paragraph

The previous document built a Workload Domain out of three vSphere Clusters, one per rack pair or spread across shared racks. This document activates vSphere Supervisor across all three of those clusters at once — each cluster becomes a vSphere Zone, and every zone does double duty as both a management zone and a workload zone. The payoff: Supervisor’s own control plane survives the loss of an entire zone, not just a single host, because its three control plane VMs are spread one-per-zone instead of stacked in one place.

Design decisions at a glance

That last row matters operationally: the three-zone model is not what a workload domain gets by default. A workload domain defaults to a single zone; three-zone Supervisor has to be explicitly opted into and activated afterward.

The zone model itself

Reading the diagram top to bottom:

• Workload Domain — the outer boundary, with its own vCenter and NSX, exactly as in the single-zone model.
• Three vSphere Zones, each labeled “(Management and Workload)” — each zone maps to its own vSphere Cluster. This is the direct activation of the three-cluster compute design from the previous document: Cluster 1 becomes Zone 1, Cluster 2 becomes Zone 2, Cluster 3 becomes Zone 3.
• vSphere Supervisor spans all three zones — this is the key visual difference from the single-zone model. Supervisor isn’t contained inside one zone’s boundary; it’s one logical construct stretched across all three, shown as the orange boundary overlapping all three clusters.
• Three Control Plane VMs, one per zone — not three VMs in one zone. This is exactly what gives Supervisor zone-level fault tolerance: losing Zone 2 entirely still leaves Control Plane VM 1 and Control Plane VM 3 running, and Supervisor keeps quorum.
• Namespaces can span zones, not just live inside one:
  • vSphere Namespace A, running VKS Clusters, stretches across all three zones — a guest Kubernetes cluster’s nodes can be placed in any of the three zones.
  • vSphere Namespace B, running plain vSphere Pods, is scoped to Zone 1 only.
  • vSphere Namespace C, running Virtual Machines, spans Zones 2 and 3.
  The point isn’t the specific split shown — it’s that a namespace’s resource pool isn’t locked to a single zone the way it was in the single-zone model. This is the practical benefit of “Multi vSphere Cluster Zones”: easier capacity expansion, because adding a cluster to a zone (or spreading a namespace across zones) doesn’t require rebuilding anything.
• A single Cross-zone Datastore at the bottom, with a connection from every zone — not three separate per-zone datastores. This is where “cross-zone” actually shows up architecturally, and it’s covered in detail next.

Storage: the cross-zone datastore

This is the storage counterpart to the three-zone compute model, and it’s a deliberate departure from the single-zone design:

• In the single-zone model (the brownfield document), storage was a zonal datastore — local to one zone, not shared, protecting only at the cluster level within that one zone.
• In this three-zone model, storage is a cross-zone datastore — a single logical datastore that spans all three vSphere Zones. Protection now happens at the infrastructure level: if an entire zone fails, the datastore itself survives, because it was never scoped to just that one zone in the first place.

Why this matters in practice: a zonal datastore ties a namespace’s storage to wherever that one zone happens to be. A cross-zone datastore means a namespace whose workloads span multiple zones (like Namespace A’s VKS Clusters, or Namespace C’s VMs above) isn’t left with storage that only half-covers where its workloads actually run.

Networking: Centralized Connectivity Model

This is the same Centralized Connectivity Model covered in full detail in the previous document (shared Tier-0 Gateway, per-tenant Centralized Transit Gateway, VPCs) — it’s referenced here rather than re-explained, because it’s exactly what connects this three-zone Workload Domain to the outside world. The connectivity model doesn’t change when the compute layer goes from one zone to three; the workload domain still presents as one tenant-facing network boundary regardless of which zone a given workload actually lands in.

The next diagram shows precisely how that connectivity model plugs into Supervisor’s own components.

Putting it together: NSX VPC networking model

NSX VPC Networking is the default Supervisor networking model when Supervisor is activated during workload domain creation. It provides the richest alignment with VCF Automation and the most comprehensive tenant-style networking workflow:

• Provides the broadest capability set for environments managed with VCF Automation.
• Uses NSX VPC constructs that map naturally to tenant and project-style operational models.
• Allows VPCs to be created in NSX projects that correspond to consumption boundaries.
• Uses connectivity profiles to define common northbound connectivity patterns for VPCs.

Reading the diagram left to right:

• vSphere Supervisor (left) owns the NSX System Project, which contains a single System Namespace and its Default VPC. That Default VPC’s VPC Gateway connects up to a Default Transit Gateway, and down through a VPC Private Subnet to the three Control Plane VMs — this is Supervisor’s own system-level networking, entirely separate from any tenant’s.
• VCF Automation Organization 1 (right) owns NSX Project 1, which is where actual tenant consumption happens. Two vSphere Namespaces each get their own VPC — VPC 1 (namespace 1) and VPC 2 (namespace 2) — each with its own VPC Gateway connecting up to the shared Transit Gateway.
• Inside each VPC, a VPC Private Transit Subnet reaches the actual workloads — two VKS Clusters in VPC 1, two vSphere pods in VPC 2 — while a separate VPC Private Subnet reaches a plain vm. Transit traffic and private-subnet traffic are kept on distinct paths even within the same VPC.
• This is the NSX Project boundary in action: VPC 1 and VPC 2 map directly to the two namespaces’ consumption boundaries — this is what “VPCs created in NSX projects that correspond to consumption boundaries” looks like concretely, one VPC per tenant-facing namespace, not one shared network for both.
• At the bottom, NSX Manager and vCenter — the same management components from every other diagram in this project — connect down into the Management Network (distributed port group), which stays completely separate from the VPC/Transit Gateway paths above it.

Why this is the default, not just an option: because it’s built on NSX Projects and VPCs from the ground up, this model requires no additional connectivity design work to get tenant-style isolation — creating a new namespace and a new VPC inside its own NSX Project is the isolation boundary. That’s why Broadcom ships it as the out-of-the-box choice when Supervisor is activated at workload domain creation, ahead of the Centralized Connectivity Model covered above, which is the model to reach for when VPC-style consumption isn’t the goal.

Network Requirements for Supervisor with NSX VPC Networking

The diagram above shows the shape of NSX VPC networking; this section gives the concrete numbers — how many IPs, which CIDR sizes, which components — needed to actually build it.

1. Management Network Requirements

2. Workload Network Requirements

• Each Supervisor must use a unique Kubernetes services CIDR range — it cannot be reused across Supervisors.
• IP pools are created per VPC, matching that VPC’s private CIDR range, and are never shared across Namespaces.

3. VPC Network Requirements

Tier-0 Uplink IP count depends on the Edge configuration: - 1 IP — no Edge redundancy. - 4 IPs — BGP with Edge redundancy (2 IP addresses per Edge). - 3 IPs — static routes with Edge redundancy.

The Edge Management IP/subnet/gateway and the Uplink IP/subnet/gateway must all be unique.

VPC Connectivity Profile, additionally: if the gateway connection type is set to Centralized Gateway, External IP Blocks must be configured and Auto SNAT must be enabled.

Requirements

Universal Supervisor requirements (apply regardless of zone count): - vSphere HA enabled on every cluster. - vSphere DRS enabled, Fully Automated or Partially Automated. - Zone names and Supervisor names must be RFC 1123 compliant (lowercase, alphanumeric, hyphens, starting with a letter or number). - A compatible vSphere Zone must be available at activation time. - Storage policies defined for control plane VMs, ephemeral disks, and the image cache.

Requirements specific to the three-zone model: - Three available vSphere Zones, none of them already assigned to another Supervisor instance — each zone is dedicated, not shared with a different Supervisor deployment. - The High Availability Control Plane model is mandatory here, not optional — three zones need quorum distributed across all three, which only the HA control plane model provides. - Three-zone Supervisor is an opt-out-then-opt-in flow: a workload domain defaults to single-zone, so three-zone activation has to be explicitly chosen and performed as a Day 2 operation from the vCenter API after the workload domain already exists.

Practical recommendations: assign control plane VM IPs statically rather than relying on external DHCP; and if Supervisor is deployed on a vSAN stretched cluster, place control plane VMs on a management network that’s extended across the availability zones involved, so they stay reachable no matter which zone they land in.

Where this fits for the client

• This is the direct continuation of the three-cluster Workload Domain built in the previous document — the same Cluster 1/2/3 become Zone 1/2/3 here, whether they’re laid out on shared racks or dedicated rack pairs.
• For a client running three geographically separate datacenters, this model’s real value shows up within a single site: three zones inside one West US workload domain gives that site’s Supervisor deployment protection against losing an entire cluster, independent of what’s happening at East US or West Europe.
• Combined with the Centralized Connectivity Model, tenants (departments, environments, business units — see the tenant discussion in the previous document) get one consistent network entry point into the workload domain regardless of which of the three zones their workloads actually run in.
• Because activation is a Day 2 operation, the client can build the three-cluster workload domain first, validate it, and activate three-zone Supervisor only when ready — there’s no requirement to commit to the three-zone model at initial deployment time.

Summary

Chapter 9: Stage 3 Load Balancer Option, Avi Load Balancer

Client: Strategic enterprise, 3 datacenters (West US, East US, West Europe)

Scope of this document: This is a Stage 3 option, not a Stage 4 capability. vSphere Supervisor supports two load balancers, under either supported networking stack, and this document is the solution design for the Avi Load Balancer choice — covering how it deploys, how its traffic flows, and how it fits alongside the brownfield (NSX Segment Networking) and greenfield (NSX VPC Networking) Supervisor designs already built in this project.

Reference: Avi Load Balancer with NSX One to One Workload Domain Deployment in VMware Cloud Foundation (Broadcom techdocs).

Where this decision sits: vSphere Supervisor networking

vSphere Supervisor supports two networking stacks, and both of them support the same two load-balancer choices:

This is why Avi belongs in Stage 3, alongside the zone-model documents, rather than in Stage 4: it’s a load-balancer choice made when Supervisor is activated, independent of whether the workload domain underneath it took the brownfield or greenfield path. The NSX Load Balancer is the simpler, built-in default with no separate control plane to deploy; Avi Load Balancer is the richer option — Layer 7, WAF, GSLB, and deep analytics — and is the subject of the rest of this document.

The solution in one paragraph

Avi is VMware Cloud Foundation’s load balancer for VKS and vSphere Supervisor workloads — it handles Layer 4 and Layer 7 load balancing, web application security, and automatic scaling. It has two parts: Avi Controllers, the control plane, deployed and lifecycle-managed by VCF Operations (which also handles their service accounts, password rotation, and certificate management automatically); and Avi Service Engines (SEs), the data plane, which actually carry traffic and can be deployed into any workload domain cluster. The one design decision that shapes everything else is how many Avi Controller clusters to deploy relative to how many NSX Manager instances exist — one shared cluster, or one dedicated cluster per workload domain.

The core design choice: shared vs. dedicated Avi Controllers

This maps directly to how NSX itself is deployed:

Neither is universally “better” — it’s a direct consequence of how NSX was already deployed for each workload domain. Avi follows NSX’s tenancy pattern, it doesn’t set its own.

The 1:1 deployment model

This is the fully isolated version of the model, and it’s the one worth designing for by default in a multi-tenant enterprise environment:

• The Management Domain hosts everything control-plane-related, for every workload domain — but kept strictly segregated by color/ownership in the diagram: WLD01’s vCenter, NSX Manager instance, and 3-node Avi Controller cluster are entirely separate objects from WLD02’s equivalents. They happen to live in the same physical Management Domain, but there is no shared instance between them.
• Each Workload Domain runs its own workloads — a mix of VKS clusters (the helm-wheel icons) and plain VMs — and its own Avi Service Engines, deployed locally into that workload domain’s cluster. WLD01’s SEs only ever carry WLD01’s traffic.
• Why this matters operationally: a certificate rotation, a password change, or a controller upgrade for WLD01’s Avi Controllers has zero effect on WLD02. This is the same “blast radius” thinking already applied to NSX Manager and vCenter elsewhere in this project’s designs — Avi just follows the same rule.

Requirements that apply regardless of shared or dedicated: - Avi must be deployed with a 1:1 relationship to its NSX Manager — a single Avi Controller cluster is never split across two different NSX Manager instances. - Avi Load Balancer must be integrated with NSX before Supervisor activation, not after — this is a sequencing requirement, not just a configuration step. - A 3-node Avi Controller cluster is the standard for availability, sized Small, Large, or X-Large depending on scale. - Controllers are deployed in the Management Domain via VCF Operations, with DRS anti-affinity rules automatically configured so the three controller nodes don’t end up on the same host.

Load balancing for vSphere Supervisor: Centralized Transit Gateway

This diagram answers a specific, practical question: once Avi is deployed, how does its traffic actually get from a client to a VKS workload, and how do the Avi Service Engines themselves get managed? The answer uses two separate paths that only meet at the Tier-0 Gateway.

The data path (left side, blue): a Client request hits the Physical Network, routes to the Tier-0 Gateway, and from there down to the Centralized Transit Gateway — the same Centralized Connectivity Model construct covered earlier in this project, now carrying Supervisor’s VPC traffic specifically. From the Transit Gateway, traffic reaches the VPC Gateway, which fans out over VPC Subnets to the actual VKS Workloads (VKS cluster nodes and plain VMs). The same VPC Gateway also connects to a VPC Services Subnet, which is how traffic actually reaches the Avi Service Engines doing the load balancing.

The management path (right side, green): the Tier-1 Gateway — a separate branch off the same Tier-0 — connects down to an AVI SE Mgmt Overlay Segment, an NSX overlay segment dedicated purely to management connectivity between the Avi Controllers and the Avi SEs. Broadcom’s design explicitly allows an alternative here too: an NSX VLAN segment can be used for this management connectivity instead of an overlay segment, if that fits the client’s existing network design better.

Why two paths instead of one: this keeps Service Engine data traffic (client requests reaching real workloads) and Service Engine management traffic (the Avi Controllers configuring and monitoring those same Service Engines) on genuinely separate networks. A data-path congestion event doesn’t interfere with the control plane’s ability to manage the Service Engines, and vice versa.

Where Avi fits across the VCF Fleet

This is the same three-region VCF Fleet from the Fleet Architecture design, with Avi’s two components dropped into place at every site:

• AVI Controllers — added to the Management Domain Cluster of every VCF Instance, alongside the vCenter and NSX Manager clusters already there. This holds regardless of whether an instance is the fleet anchor (US WEST, with the full fleet-wide services grid) or an instance-level peer (US EAST, EUROPE WEST) — every site runs its own Avi Controller cluster, matching the per-site, per-NSX-Manager 1:1 relationship covered earlier in this document.
• Service Engines — added to the Workload Domain Cluster of every VCF Instance, alongside the NSX Edge and Supervisor VM Cluster components. This is the data plane, placed where it actually does its work: in the same workload domain cluster as the VKS and Supervisor workloads it’s load balancing.

Why this repeats identically at all three sites: Avi Controller lifecycle and Service Engine placement aren’t fleet-level services — they don’t get centralized in US WEST the way VCF Operations, VCF Automation, and the License Server are. Each site’s Avi Controller cluster manages that site’s own Service Engines only, which is consistent with the “blast radius” isolation already designed into every other management-plane component in this project.

Requirements and recommendations

Critical requirements: - Enable NSX Cloud when Avi is used as the load balancer for VPC-based Supervisor networking (the NSX VPC model covered in the Greenfield vSphere Supervisor Design). - Avi’s 1:1 relationship with NSX Manager (stated above) is not optional — it’s a hard requirement, not just the recommended pattern. - Avi must be integrated with NSX before Supervisor is activated.

Recommendations: - Deploy the 3-node Avi Controller cluster in the Management Domain, managed through VCF Operations, so certificate/password/service-account lifecycle is automated rather than manual. - Enable VPC mode in the NSX Cloud Account for project discovery — for VPC-based Supervisor deployments, VPC mode is actually on by default when Avi is deployed through VCF Operations. - Enable DHCP addressing for Service Engines so SE IPs are provisioned automatically rather than requiring manual allocation per SE. - In VPC deployments specifically, Service Engines communicate with the VPC over the VPC Services Subnet shown in the diagram — a separate NSX VLAN or overlay segment isn’t needed for that data path, only for SE management.

Where this fits for the client

• Given the client’s three separate, independently operated datacenters (West US, East US, West Europe) and the NSX-per-workload-domain pattern already established in this project’s earlier import and greenfield designs, the 1:1 model is the natural fit — each workload domain, at each site, gets its own Avi Controller cluster, matching its own dedicated NSX Manager instance.
• This is a Stage 3 choice, not a Stage 4 dependency — it doesn’t need to be part of the initial VKS rollout. A workload domain can activate Supervisor with the built-in NSX Load Balancer first, on either the brownfield (NSX Segment Networking) or greenfield (NSX VPC Networking) path, and adopt Avi later without redesigning the workload domain itself. Nothing about the zone model, storage, or connectivity design already chosen for that workload domain changes based on which load balancer sits underneath Supervisor.
• Because Avi Controller lifecycle (passwords, certificates, service accounts) is automated through VCF Operations, adding Avi doesn’t add a new manual operational burden per site — it plugs into the same VCF Operations footprint already managing everything else across the three datacenters.

Summary

Chapter 10: Stage 4, F5 to Avi Load Balancer Migration

Client: Strategic enterprise, 3 datacenters (West US, East US, West Europe)

Scope of this document: The final document in this series, and a Stage 4 design. It answers one question: how does the client retire its existing F5 BIG-IP estate — described in Existing State — External and Global Traffic Management — and replace it with VMware Avi Load Balancer (formerly NSX Advanced Load Balancer), the same Avi platform already designed as a Stage 3 load-balancer option for VKS and vSphere Supervisor in vSphere Supervisor Load Balancer Option: Avi Load Balancer. This document extends Avi beyond Supervisor to the client’s entire application estate — the piece that actually makes Stage 4 a complete private cloud operating model.

1. The solution in one paragraph

F5 does two jobs today: BIG-IP DNS/GTM decides which of the three datacenters serves a user, and BIG-IP LTM load-balances inside the selected datacenter, in front of plain vSphere 7/8 VM workloads. Avi replaces both jobs with its own two-tier architecture: Avi GSLB takes over the global “which site” decision, and Avi Virtual Services running on Avi Service Engines take over the local “which server” decision. Nothing about the shape of the traffic-management problem changes — it is still a global tier sitting above a local tier — only the vendor and the underlying architecture change, from hardware-appliance-based F5 to the software-defined, Controller/Service-Engine model Avi already uses everywhere else in this project.

2. Mapping F5 to Avi

Every F5 concept the client operates today has a direct Avi equivalent. This table is the migration Rosetta Stone — refer back to it throughout the rest of this document:

Do not treat this as a one-to-one automatic conversion. A few F5 features need real design decisions rather than a straight lookup — see Section 7, Migration cautions.

3. Target-state traffic flow

This is the direct replacement for the flow in the existing-state document, hop for hop:

1. DNS Query — a user’s device resolves the application name against Avi GSLB, which takes over DNS delegation from F5 BIG-IP DNS/GTM.
2. Best Site Selected — Avi GSLB evaluates site health and applies a GSLB policy (active/active, active/passive, geo-based, priority-based, or ratio-based) and returns the best region’s Virtual Service — the same job GTM did, just on Avi.
3. VIP / Service Engines — the regional Avi Virtual Service, running on distributed Avi Service Engines, terminates the connection, applies SSL, checks pool member health, and load-balances to the application. The application itself doesn’t have to change on day one: it can still be a plain vSphere VM, exactly as it is today, and later become a VCF or VKS workload without another redesign of this flow.

Where the Avi Controller fits: it is deliberately not drawn as a traffic hop. The Controller cluster is the control plane — it configures Virtual Services, runs health monitors, and pushes configuration to Service Engines — but it never sits in the path of a live request. Only the Service Engines carry traffic, which is exactly the control-plane/data-plane split VMware’s Avi architecture is built around.

4. Component design per region

Each of the three regions is built identically — there is no “primary” Avi site the way F5 GTM sometimes centralizes configuration. Per region:

Above all three regions sits Avi GSLB, configured with a single GSLB Service (for example app.company.com) whose pool members are the three regions’ Virtual Services. GSLB only ever talks to each region’s Controller cluster for health and configuration — it never touches application data traffic directly.

Separating Service Engines into Production, Non-Production, and VKS/Kubernetes groups mirrors the same isolation principle used everywhere else in this project (the “blast radius” thinking applied to NSX Manager, vCenter, and Avi Controllers in the earlier documents): a capacity problem or a bad deployment in one group doesn’t starve the others.

5. Migration approach: seven phases

F5 and Avi run in parallel for the entire migration — there is no big-bang cutover. F5 stays in production until every application has been individually validated on Avi and moved over.

6. Why replace F5 with Avi

VCF 9.1 recommends Avi specifically for environments that need advanced ADC capability — GSLB, deep traffic analytics, complex traffic management, and application security (WAF) — which describes this client’s current F5 use exactly. For VKS workloads, Avi is the recommended ingress path via AKO whenever an application needs Layer 7 balancing, WAF, or DNS integration beyond what a basic in-cluster ingress controller provides.

7. Migration cautions

A handful of F5 features don’t translate automatically and need explicit validation during Phase 1 and Phase 3:

8. Where this fits for the client

• This document closes the loop on the existing-state traffic management baseline: F5 BIG-IP DNS/GTM and LTM are the last piece of the client’s original three-site estate still standing once the rest of this project’s designs (fleet architecture, brownfield import, greenfield workload domains, Supervisor design, and the Avi Load Balancer option for VKS/Supervisor) are in place.
• Because Avi is already the load-balancing layer chosen for VKS and vSphere Supervisor as a Stage 3 option, replacing F5 with Avi for general application traffic means the client ends up with one load-balancing platform across the entire estate — VM workloads, VCF, and VKS — instead of F5 for legacy traffic and Avi for platform traffic running side by side indefinitely.
• The phased, wave-based migration approach means this is not a dependency that blocks any earlier stage of the modernization journey — F5 keeps working exactly as it does today for as long as it’s needed, and each application moves to Avi on its own schedule.
• Given the client’s three-datacenter footprint (West US, East US, West Europe), the three-region Avi component design in Section 4 maps directly onto existing site boundaries — no new site, no new network dependency, just a like-for-like regional structure with Avi in place of F5.

Summary

Document Map — How the Chapters Fit Together

Each stage is a superset of the one before it: moving forward never requires rebuilding what came before, and the client’s three sites do not need to progress through the stages in lockstep.

Reference Table

Every external URL cited across this document’s source material, consolidated in one place. All 23 links were checked on 2026-07-08 and are live. Two Dell links return an automated-client block (HTTP 403) to a plain, headerless request — this is Dell’s bot protection, not a broken link; both were independently confirmed reachable and correct via a full browser-style fetch.

Design library and how-to documentation (Broadcom TechDocs)

Hardware compatibility references

Source reference diagrams (Broadcom original assets)

These are the original Broadcom PNG/SVG master images each corresponding draw.io diagram in this document was recreated from — kept here for provenance and future re-verification against the source.