June 2026
A Technical Reference and Lab Validation Report
| Platform | VMware Cloud Foundation / vSphere Supervisor with NSX |
| Networking model | NSX Classic Model (Tier-0 / Tier-1 / segments — not the NSX VPC model) |
| Subject | The Workload Network NAT mode (Enabled vs. Disabled) chosen when activating the Supervisor, and per-namespace via Override network |
| Lab | LAB-POC (NSX Manager nsx01.vcf.nicolescu.org) |
| Validated | 15 June 2026 |
| Audience | Network architects, platform engineers, and operations teams |
This document covers activating a vSphere Supervisor using the NSX Classic Model and, specifically, the NAT mode decision made while configuring the Workload Network.
Where the NAT mode setting appears (two places):
The lab namespace used in this report is literally named
dev-no-override— it does not override the Supervisor network settings, so it inherits the Supervisor’s NAT mode. See Section 3.
The two modes behave as follows:
NAT Enabled — NSX hides the workloads behind a shared egress (SNAT) IP address. The outside world sees that egress IP, not the real workload IP. This keeps upstream routing simple, at the cost of source-IP visibility.
NAT Disabled (“routed” or “no-NAT” mode) — workloads keep their real source IP all the way out. This gives full visibility and clean end-to-end routing, but the physical network must know how to route back to the Namespace Network.
The single concrete difference we measured between the two modes, on the same cluster, is the set of SNAT rules on each namespace’s Tier-1 gateway and the matching egress IP routes that those rules advertise upward:
| NAT Enabled | NAT Disabled | |
|---|---|---|
| Per-namespace SNAT rule | Present (translates to an egress IP) | Absent |
| Egress CIDR required | Yes (10.88.32.0/21) |
No |
| Egress IPs in NSX routing table | Yes (10.88.32.1/32,
10.88.32.2/32) |
No |
| Source IP seen externally | The egress IP | The Kubernetes Node IP |
Everything else — the namespace segments, the LoadBalancer VIPs, the Ingress CIDR — behaved identically in both captures. This is shown with evidence in Section 5 and Section 6.
Activating a Supervisor with the NSX Classic Model asks you to supply several distinct IP ranges on the Workload Network screen. They each have one job. Keeping them straight is the key to understanding NAT behaviour. The same fields reappear when you create a namespace with Override Supervisor network settings checked.
Each subsection below pairs the official VMware field definition (shown as a quoted block) with a plain-English explanation and the values used in this lab.
NAT Mode — Unchecking NAT mode will result in an NSX-T topology for Namespaces in routed mode. This mode enables reachability to workload IP addresses directly from external networks.
In other words, the NAT checkbox is the single switch that decides between “hide workloads behind a translated egress IP” (checked) and “route the real workload IPs” (unchecked). It is presented in two places — at Supervisor activation (the Supervisor-wide default) and per-namespace under Override network — but the underlying NSX behaviour it controls is identical in both. The rest of this section describes the address ranges that switch acts upon.
Namespace Network — Internal CIDR blocks from which IPs for pods will be allocated. They cannot overlap with IPs of Supervisor Management components (vCenter, NSX, ESXi, Management DNS, NTP) and should not overlap with other datacenter IPs communicating with pods.
Namespace Network: 10.55.16.0/20
Subnet Prefix: /24
Think of the Namespace Network as a large pool of addresses that NSX carves up, one slice per Supervisor namespace. The Subnet Prefix decides how big each slice is.
With a /20 pool and a /24 prefix, NSX can
create 16 namespace segments:
10.55.16.0/24 10.55.20.0/24 10.55.24.0/24 10.55.28.0/24
10.55.17.0/24 10.55.21.0/24 10.55.25.0/24 10.55.29.0/24
10.55.18.0/24 10.55.22.0/24 10.55.26.0/24 10.55.30.0/24
10.55.19.0/24 10.55.23.0/24 10.55.27.0/24 10.55.31.0/24
Each namespace gets one /24 (~254 usable addresses).
Subnet Prefix — The Namespace subnet prefix defines the IP subnet created for each Supervisor Namespace segment. For example, defining a
/24prefix would result in a Supervisor Namespace segment with an IP subnet of 254 IP addresses to allocate to workloads.
Subnet Prefix sizing — how many addresses per namespace?
| Subnet Prefix | Usable IPs per namespace | Namespaces from a /20 pool |
When to use |
|---|---|---|---|
/28 (default) |
~14 | 256 | Many small namespaces |
/26 |
~62 | 64 | Medium namespaces |
/24 (this lab) |
~254 | 16 | Fewer, larger namespaces |
Smaller prefix number = bigger subnet = more IPs per namespace but fewer namespaces overall. The default of
/28favours a large number of namespaces; this lab used/24for headroom per namespace.
Service CIDR — Internal CIDR block from which IPs for Kubernetes ClusterIP Services will be allocated. It cannot overlap with IPs of Supervisor Management components (vCenter, NSX, ESXi, Management DNS, NTP) and should not overlap with other datacenter IPs communicating with pods.
Service CIDR: 10.66.64.0/20
These are virtual IPs used only inside Kubernetes to
front a group of pods (ClusterIP services). A service IP such as
10.66.64.10 is never a real pod — it is a stable front door
that load-balances to the real pods behind it:
Service IP 10.66.64.10 -> pods 10.55.17.21, 10.55.17.22, 10.55.17.23
This range must not overlap with any management, DNS, NTP, ESXi, vCenter, NSX, or datacentre network.
Supervisor Control Plane size sets the default Service CIDR. When the Supervisor is deployed, the chosen control-plane size pre-populates a default Service CIDR (which you can override). Larger control planes default to a larger service range:
| Control Plane Size | CPUs | Memory | Storage | Default Service CIDR | Approx. service IPs |
|---|---|---|---|---|---|
| Small | 4 | 16 GB | 48 GB | 10.96.0.0/23 |
~510 |
| Medium | 8 | 24 GB | 48 GB | 10.96.0.0/22 |
~1022 |
These defaults apply to the Supervisor. Note this is a different Service CIDR from the VKS guest cluster service CIDR (
10.96.0.0/12) seen in the inventory reports — the guest cluster manages its own service range internally (see Section 7).
Ingress CIDRs — External CIDR blocks from which IPs for Kubernetes Ingresses and LoadBalancer-type Services will be allocated. These CIDRs come from the external network’s IP space. The Supervisor control plane allocates 1 common Kubernetes Ingress IP per namespace (all Ingresses in the same namespace share a common IP), and 1 IP per LoadBalancer-type Service (each LoadBalancer Service is allocated a unique IP).
Ingress CIDR: 10.77.64.0/21
This range supplies the externally reachable IPs for
Kubernetes Ingress objects and
Service type: LoadBalancer. It is used in
both NAT modes and must be routable from the outside.
Examples observed in the lab:
Kubernetes API VIP: 10.77.64.2 / 10.77.68.1
Application LoadBalancer: 10.77.64.3 (or 10.77.68.3)
SSH-test LoadBalancer: 10.77.64.4
Allocation rule of thumb: one shared Ingress IP per namespace, plus one dedicated IP per LoadBalancer Service. Size the Ingress CIDR for the total number of namespaces and LoadBalancer services you expect.
Egress CIDRs — External CIDR blocks used for translating internal Pod IPs to an external IP via SNAT rules, allowing pods to reach the external network. These CIDRs come from the external network’s IP space. The Supervisor control plane allocates 1 egress IP to each namespace.
Egress CIDR: 10.88.32.0/21 (this lab)
When NAT is enabled, workload traffic leaving the cluster is
translated to an IP from this range. The control plane
assigns one egress IP per namespace — which is exactly
what the lab shows: 10.88.32.1 for the system namespace and
10.88.32.2 for the dev-no-override namespace
(see Section 5.1). When NAT is disabled,
no Egress CIDR is configured at all.
A Supervisor was activated with the NSX Classic
Model. Both NAT modes were validated against the same
physical lab by changing only the Workload Network NAT
mode at the Supervisor level and re-activating; the test
namespace dev-no-override inherited that Supervisor-wide
setting (it does not use Override network). The facts
below are taken directly from the NSX and Kubernetes captures.
Source: nsx-network-config-report-20260615-170453-nat.txt
| Component | Value |
|---|---|
| NSX Manager | https://nsx01.vcf.nicolescu.org |
| vSphere Distributed Switch | lab-cl01-vds01 |
| Edge Cluster | LAB-POC-EDGE-CLUSTER-01 (2 edge VMs) |
Edge node nsx01-eg01 |
192.168.200.34 |
Edge node nsx01-eg02 |
192.168.200.35 |
| Tier-0 Gateway | LAB-POC-T0-GW-01 |
| Tier-1 Gateways | 2 (one per namespace shown) |
| DNS Servers | 192.168.200.7, 192.168.200.8 |
| Setting | Value |
|---|---|
| External (uplink) interfaces | 192.168.204.11/24, 192.168.204.12/24 (VLAN
204) |
| Default route | 0.0.0.0/0 → 192.168.204.1 (static, admin distance
1) |
| BGP | Local AS 65000, ECMP on — enabled but no
neighbours configured in this lab |
| Route redistribution | Enabled (T0/T1 connected, static, NAT, LB VIP/SNAT) |
Lab note: northbound reachability in this environment is provided by the static default route to
192.168.204.1. BGP is switched on with a local AS but has no peers, so route advertisement to a physical router was not exercised here. In production, the Namespace/Ingress/Egress ranges would typically be advertised via BGP from the Tier-0 (see Section 9).
Source: “Segments” table in the NSX reports.
| Segment role | Gateway / Subnet | Attached Tier-1 |
|---|---|---|
| Supervisor system namespace | 10.55.16.1/24 |
domain-c10:7e0820ed-… |
Namespace dev-no-override |
10.55.17.1/24 |
t1_…_dev-no-override_rtr |
VKS vnet segment (dev-no-override) |
10.55.18.1/24 |
t1_…_dev-no-override_rtr |
In the NSX Classic Model each Supervisor namespace gets its own dedicated Tier-1 gateway and overlay segment(s). The namespace
dev-no-overridewas created without the Override Supervisor network settings option, so it draws its CIDRs and NAT mode from the Supervisor defaults. Had it been created with Override network, it would have its own NAT mode and CIDRs while keeping its own Tier-1.
dev-cluster-01Source: k8s-ip-report-20260615-170232-nat.txt
and …-135323-no-nat.txt
A six-node Tanzu/VKS guest Kubernetes cluster was deployed to generate real traffic:
10.55.18.2 –
10.55.18.7 — these come from the Supervisor
Namespace Network (segment 10.55.18.0/24).192.168.156.0/20
(network base 192.168.144.0), carved into per-node
/24s such as 192.168.144.0/24 …
192.168.150.0/24 — these belong to the guest
cluster, not to NSX.10.96.0.0/12 —
internal to the guest cluster.Two LoadBalancer services were running to test ingress in both modes:
| Service | Cluster IP | External IP (NAT) | External IP (no-NAT) | Port |
|---|---|---|---|---|
assembly-lb / assembly-service |
(internal) | 10.77.64.3 |
10.77.68.3 |
8080/TCP |
network-test / ol9-network-test-lb |
(internal) | 10.77.64.4 |
10.77.64.4 |
22/TCP |
This gives us two clean layers to reason about — the NSX namespace
layer (10.55.x) and the guest pod layer
(192.168.x) — which matters greatly for NAT, as explained
in Section 7.
The clearest way to see the difference is to follow one packet from a
workload to an external DNS server (192.168.200.7).
Pod / namespace workload External DNS
10.55.17.25 192.168.200.7
| ^
| src=10.55.17.25 dst=192.168.200.7 |
v |
NSX Tier-1 --- SNAT ---> src rewritten to 10.88.32.x
| |
+------------------------------------------+
The DNS server sees the request coming FROM 10.88.32.x
It never sees 10.55.17.25.
Return traffic comes back to the egress IP and NSX translates it back to the original workload — the workload is unaware any translation happened.
Pod / namespace workload External DNS
10.55.17.25 192.168.200.7
| ^
| src=10.55.17.25 dst=192.168.200.7 |
v |
NSX Tier-1 (NO translation) ---> Tier-0 ---> |
| |
+------------------------------------------+
The DNS server sees the request coming FROM 10.55.17.25 (the real IP).
For the reply to return, the upstream network MUST route 10.55.16.0/20
back toward the NSX Tier-0.
In one sentence: NAT enabled hides workloads behind SNAT addresses; NAT disabled routes the real workload networks through NSX Tier-0.
This is the heart of the change. Below are the actual NAT rules programmed on the namespace Tier-1 gateways in each mode, copied verbatim from the captures.
Source: nsx-network-config-report-20260615-170453-nat.txt
Tier-1 — Supervisor system namespace
(domain-c10:7e0820ed-c75d-45d6-973a-7d55fb2adfed):
| Rule | Action | Source | Destination | Translated | Pool Range |
|---|---|---|---|---|---|
nat-domain-c10:7e0820ed-… |
SNAT | any | any | 10.88.32.1 |
10.88.32.1 – 10.88.39.254 |
no_nat-…-10-55-16-0-20-10-55-16-0-20 |
NO_SNAT | 10.55.16.0/20 |
10.55.16.0/20 |
– | – |
no_nat-…-10-55-16-0-20-10-77-64-0-21 |
NO_SNAT | 10.55.16.0/20 |
10.77.64.0/21 |
– | – |
NAT summary: SNAT=1, DNAT=0, Total=3
Tier-1 — dev-no-override namespace:
| Rule | Action | Source | Destination | Translated | Pool Range |
|---|---|---|---|---|---|
nat-dev-no-override |
SNAT | any | any | 10.88.32.2 |
10.88.32.1 – 10.88.39.254 |
no_nat-dev-no-override-10-55-16-0-20-10-55-16-0-20 |
NO_SNAT | 10.55.16.0/20 |
10.55.16.0/20 |
– | – |
no_nat-dev-no-override-10-55-16-0-20-10-77-64-0-21 |
NO_SNAT | 10.55.16.0/20 |
10.77.64.0/21 |
– | – |
NAT summary: SNAT=1, DNAT=0, Total=3
Source: nsx-network-config-report-20260615-135313-no-nat.txt
Tier-1 — Supervisor system namespace:
| Rule | Action | Source | Destination | Translated | Pool Range |
|---|---|---|---|---|---|
no_nat-…-10-55-16-0-20-10-55-16-0-20 |
NO_SNAT | 10.55.16.0/20 |
10.55.16.0/20 |
– | – |
no_nat-…-10-55-16-0-20-10-77-64-0-21 |
NO_SNAT | 10.55.16.0/20 |
10.77.64.0/21 |
– | – |
NAT summary: SNAT=0, DNAT=0, Total=2
Tier-1 — dev-no-override namespace:
| Rule | Action | Source | Destination | Translated | Pool Range |
|---|---|---|---|---|---|
no_nat-dev-no-override-10-55-16-0-20-10-55-16-0-20 |
NO_SNAT | 10.55.16.0/20 |
10.55.16.0/20 |
– | – |
no_nat-dev-no-override-10-55-16-0-20-10-77-64-0-21 |
NO_SNAT | 10.55.16.0/20 |
10.77.64.0/21 |
– | – |
NAT summary: SNAT=0, DNAT=0, Total=2
Note how, compared with Section 5.1, the
single nat-… SNAT rule (translating to an
egress IP) has disappeared from each Tier-1; the two
NO_SNAT exception rules remain unchanged.
NAT ENABLED NAT DISABLED
---------------------------- --------------------------
SNAT rule 1 per namespace Tier-1 NONE
Egress IP 10.88.32.1, 10.88.32.2 ... not allocated
Egress pool 10.88.32.1 - 10.88.39.254 not configured
NO_SNAT rules 2 (unchanged) 2 (unchanged)
Rules per T1 3 2
Two takeaways:
Enabling NAT adds exactly one SNAT rule per namespace
Tier-1, each translating all egress
(any → any) to one IP from the Egress CIDR. Each namespace
gets its own egress identity (10.88.32.1 for the system
namespace, 10.88.32.2 for
dev-no-override).
The NO_SNAT rules are present in
both modes. They are exception rules that
say: “traffic from the Namespace Network to the Namespace Network
(10.55.16.0/20 → 10.55.16.0/20) and to the Ingress CIDR
(10.55.16.0/20 → 10.77.64.0/21) must not
be translated.” This keeps east-west and ingress-return traffic on its
real IPs even when NAT is enabled. When NAT is disabled they remain, but
with no SNAT rule above them they simply have nothing to
override.
The NAT rules above leave a fingerprint in the NSX Tier-0 forwarding/RIB table, because the egress IPs have to be advertised so return traffic can find them.
Source: “Routing Table RIB” in each NSX report.
The namespace segments and LoadBalancer VIPs are advertised to the Tier-0 regardless of NAT setting:
T1-connected 10.55.16.0/24 -> 100.64.0.1 (system namespace segment)
T1-connected 10.55.17.0/24 -> 100.64.0.3 (dev-no-override namespace)
T1-connected 10.55.18.0/24 -> 100.64.0.3 (VKS vnet segment)
t1l 10.77.64.x/32 (LoadBalancer / API VIPs)
t1l 10.77.68.x/32 (LoadBalancer / API VIPs)
t0s 0.0.0.0/0 -> 192.168.204.1 (default uplink)
t1n 10.88.32.1/32 -> 100.64.0.1 <-- system namespace egress IP
t1n 10.88.32.2/32 -> 100.64.0.3 <-- dev-no-override egress IP
These t1n (Tier-1 NAT) /32 host routes for
the egress IPs appear only in the NAT-enabled capture.
In the NAT-disabled capture they are absent — there are no egress IPs to
advertise.
This is the most commonly misunderstood part, so it deserves its own section.
There are two separate IP layers:
Supervisor Namespace Network : 10.55.16.0/20 (NSX-backed, what NAT mode acts on)
VKS guest cluster Pod CIDR : 192.168.156.0/20 (owned by the guest cluster CNI)
The Supervisor Workload NAT setting acts on the Namespace
Network — not on the guest Pod CIDR. NSX cannot and does not
rewrite the guest cluster’s 192.168.x pod addresses; those
are managed inside the guest cluster by its CNI (Antrea in this
lab).
So how is VKS traffic affected at all? Because VKS nodes sit
on the Namespace Network (10.55.18.2–7). What the
outside world sees depends on the guest CNI’s masquerade behaviour:
Common case — guest CNI masquerades pod traffic to the node IP:
NAT ENABLED: 192.168.156.25 (pod) -> 10.55.18.3 (node) -> SNAT -> 10.88.32.x (external sees egress IP)
NAT DISABLED: 192.168.156.25 (pod) -> 10.55.18.3 (node) ----------> 10.55.18.3 (external sees node IP)
In this common case the external network never sees
the 192.168.156.0/20 pod IP — only the node IP or the NSX
egress IP.
If the guest CNI does not masquerade, pod
traffic can leave with the real 192.168.156.x source. The
upstream network would then also need a route back to
192.168.156.0/20. This is controlled by the guest
cluster CNI, not by the Supervisor Workload NAT setting.
Bottom line: the Supervisor Workload NAT setting directly affects Namespace-Network traffic (including VKS node IPs). It affects VKS pod traffic only indirectly, and the exact behaviour is decided by the guest cluster’s CNI/masquerade configuration.
| Item | NAT Enabled | NAT Disabled |
|---|---|---|
| Namespace Network | Required | Required |
| Subnet Prefix | Required | Required |
| Service CIDR | Required | Required |
| Ingress CIDR | Required | Required |
| Egress CIDR | Required (10.88.32.0/21) |
Not used |
| Per-namespace SNAT rule | Yes | No |
| NAT rules per Tier-1 (observed) | 3 (1 SNAT + 2 NO_SNAT) | 2 (NO_SNAT only) |
| Egress host routes in RIB | 10.88.32.1/32, 10.88.32.2/32 |
none |
| Source IP seen externally | Egress / SNAT IP | Real workload / node IP |
| Upstream route to Namespace Net | Usually not required | Required |
| LoadBalancer / Ingress | Works | Works |
| Troubleshooting visibility | Lower | Higher |
| Firewall source attribution | Egress IP | Real workload IP |
| Routing complexity | Lower | Higher |
| Operational model | Hide workloads behind SNAT | Route workloads directly |
The upstream/physical network must reach the external-facing ranges only:
10.77.64.0/21 (Ingress CIDR) via NSX Tier-0
10.88.32.0/21 (Egress CIDR) via NSX Tier-0
A direct upstream route to 10.55.16.0/20 is normally
not required, because workload egress is translated to
the Egress CIDR.
The upstream/physical network must reach the Namespace Network and the Ingress CIDR:
10.55.16.0/20 (Namespace Network) via NSX Tier-0
10.77.64.0/21 (Ingress CIDR) via NSX Tier-0
This is achieved by either:
192.168.204.11/.12).In this lab the Tier-0 reaches the outside via a static default route to
192.168.204.1; BGP is enabled (AS 65000) but has no neighbours, so the namespace ranges were not advertised to a physical router during testing.
Check, in order:
[ ] Egress CIDR is configured on the Supervisor Workload Network
[ ] A SNAT rule exists on the namespace Tier-1 (NAT summary SNAT=1)
[ ] The namespace Tier-1 is connected to the Tier-0
[ ] The Tier-0 has an upstream/default route
[ ] The upstream firewall permits the Egress IP (10.88.32.x)
[ ] A return path to the Egress CIDR exists
Expected external source IP: an egress IP from
10.88.32.0/21.
[ ] The Namespace Network is routed upstream (BGP or static)
[ ] The Tier-1 advertises its connected namespace routes
[ ] The Tier-0 redistributes/advertises those routes (or static routes exist)
[ ] The upstream firewall permits Namespace-Network source IPs (10.55.16.0/20)
[ ] A return path to the Namespace Network exists
Expected external source IP: the real workload / node IP
(10.55.x.x).
Classic symptoms of a missing return route in no-NAT mode:
- Pods can send traffic out, but replies never come back
- A LoadBalancer IP is assigned but is unreachable
- Asymmetric or intermittent connectivity
[ ] The Ingress CIDR (10.77.64.0/21) is routed upstream
[ ] The Tier-0 has the LoadBalancer VIP route (t1l .../32)
[ ] The NSX LB virtual server exists and the pool has healthy members
[ ] The firewall permits the VIP and port (e.g. 10.77.64.4:22)
[ ] A return path exists
Choose NAT Enabled when:
- The physical network cannot (or should not) route to the Namespace Network
- You want the simplest possible upstream routing
- You do not need external systems to see real pod/namespace IPs
- You want workloads hidden behind a small set of egress IPs
Choose NAT Disabled when:
- The Namespace Network can be made fully routable upstream
- You need real source-IP visibility (security, audit, firewall attribution)
- You want clean end-to-end routing and troubleshooting
- Your team is comfortable managing static routes or BGP on the upstream
In one line: > NAT enabled hides workloads behind SNAT addresses and keeps upstream routing simple; NAT disabled preserves real workload IPs and requires the upstream network to route the Namespace Network.
Mixing both in one Supervisor. Because NAT mode can be set per-namespace via Override Supervisor network settings, you are not limited to a single choice. A common pattern is to activate the Supervisor with the safer default (NAT Enabled) and then create selected namespaces with Override network → NAT Disabled (plus their own routable CIDRs) where real source-IP visibility is required. Each overriding namespace still gets its own dedicated Tier-1 gateway, so its NAT behaviour is fully independent of the others.
| Range | CIDR | Notes |
|---|---|---|
| Namespace Network | 10.55.16.0/20 |
16 × /24 namespace segments; used by Supervisor
namespaces, VKS nodes |
| Subnet Prefix | /24 |
~254 usable IPs per namespace segment |
| Service CIDR (Supervisor) | 10.66.64.0/20 |
Internal ClusterIP services |
| Ingress CIDR | 10.77.64.0/21 |
External LoadBalancer / Ingress / API VIPs — both modes |
| Egress CIDR | 10.88.32.0/21 |
NAT-enabled only; SNAT pool
10.88.32.1–10.88.39.254 |
| Tier-0 uplink | 192.168.204.0/24 |
VLAN 204; gateway 192.168.204.1 |
| Edge / mgmt | 192.168.200.0/24 |
Edge nodes .34/.35; DNS .7/.8 |
| VKS guest Pod CIDR | 192.168.156.0/20 |
Guest cluster only (base 192.168.144.0) |
| VKS guest Service CIDR | 10.96.0.0/12 |
Guest cluster only |
Observed namespace segments (both modes):
| Segment | Tier-1 |
|---|---|
10.55.16.0/24 (system) |
domain-c10:7e0820ed-… |
10.55.17.0/24 (dev-no-override) |
t1_…_dev-no-override_rtr |
10.55.18.0/24 (VKS vnet) |
t1_…_dev-no-override_rtr |
All evidence in this document is reproducible from the following
captures, taken on 15 June 2026 against NSX Manager
nsx01.vcf.nicolescu.org.
k8s-ip-report-20260615-170232-nat.txt
Generated 17:02:32 — VKS dev-cluster-01, nodes
10.55.18.2–7, LB VIPs 10.77.64.3/.4.nsx-network-config-report-20260615-170453-nat.txt
Generated 17:04:53 — SNAT rules to 10.88.32.1/.2;
egress host routes in RIB.k8s-ip-report-20260615-135323-no-nat.txt
Generated 13:53:23 — same cluster, LB VIPs 10.77.68.3 /
10.77.64.4.nsx-network-config-report-20260615-135313-no-nat.txt
Generated 13:53:13 — NO_SNAT rules only (SNAT=0); no egress CIDR or
egress routes.This is the inside-the-cluster view of the same lab,
captured with NAT enabled from VKS guest cluster
dev-cluster-01. The full output is reproduced at the end of
this appendix; the source file is k8s-ip-report-20260615-170232-nat.txt.
The NSX captures in Section 5 and Section 6 show the network’s view — the SNAT rules and routes. This Kubernetes report shows the workload’s view. Reading them together is what lets you trace a single packet end-to-end and troubleshoot with confidence. The two views are deliberately complementary.
1. The two IP layers are visible side by side — exactly as
designed. - Node IPs are all in
10.55.18.x → the Supervisor Namespace
Network (NSX segment 10.55.18.0/24, part of
10.55.16.0/20). - Pod IPs are in
192.168.144.x – 192.168.150.x → the
VKS guest Pod CIDR (192.168.156.0/20,
network base 192.168.144.0). - This is the two-layer concept from
Section 8 made concrete. NSX never sees the 192.168.x
pod IPs; it only ever acts on the 10.55.18.x node IPs.
2. Each node owns a distinct /24 slice of the
pod CIDR. Standard Kubernetes node IPAM gives every node its
own /24 out of the cluster pod CIDR — six nodes, six slices
(192.168.144.0/24 … 192.168.150.0/24). A pod’s
IP therefore tells you which node it is on.
3. Some pods show a 10.55.18.x IP, not a
192.168.x IP — these are host-network pods.
Components such as antrea-agent, kube-proxy,
etcd, kube-apiserver,
kube-controller-manager, kube-scheduler,
vsphere-csi-node, and the auth/cloud-provider pods run with
hostNetwork: true and therefore share the node’s
namespace-network IP. This matters for NAT: their egress is
translated exactly like node traffic (SNAT to an egress
IP), because they already live on 10.55.18.0/24.
Application pods (assembly-deployment,
coredns, metrics-server,
network-test, CSI controller) use real
192.168.x pod IPs and reach NSX only after the guest CNI
masquerades them to the node IP.
4. Control-plane vs. worker split is readable from the
data. The three …dpwx4… nodes
(10.55.18.2/.6/.7) host etcd,
kube-apiserver, kube-scheduler, and
kube-controller-manager → control-plane
nodes. The three …np-01-cmrnj… nodes
(10.55.18.3/.4/.5) carry the application replicas →
worker nodes. A balanced 3 + 3 cluster.
5. The Service CIDR here (10.96.0.0/12) is the
guest cluster’s, not the Supervisor’s. ClusterIPs like
10.96.0.1 (kubernetes) and
10.96.0.10 (kube-dns) come from this guest
range. It is independent of the Supervisor Service CIDR
(10.66.64.0/20) and of the Supervisor control-plane default
(10.96.0.0/23 / /22) discussed in Section 2.2.
These are separate cluster scopes that happen to share the conventional
10.96.0.0 base — do not confuse them.
6. LoadBalancer external IPs come from the Ingress CIDR — and
ingress works under NAT. assembly-service received
10.77.64.3 and ol9-network-test-lb received
10.77.64.4, both from the Ingress CIDR
10.77.64.0/21. The services’ ClusterIPs
(10.105.77.235, 10.104.102.169) are internal
only. This confirms that enabling NAT does not affect inbound
LoadBalancer/Ingress (see Section 2.3).
7. Endpoints resolve to real pod IPs, not the VIP.
assembly-service endpoints are
192.168.145.2 / 192.168.147.2 / 192.168.148.2 — the three
application replicas spread across the worker nodes. The
supervisor service endpoint 10.77.68.1 is the
Supervisor/API VIP from the Ingress range. This is how the NSX load
balancer knows which backends to forward to.
8. The most important thing is what the report does
not show: the egress IP. Nowhere in this Kubernetes
output does 10.88.32.x appear. SNAT is invisible
from inside the cluster — the workload has no idea its source
IP is being translated on the way out. That transparency is the entire
point of NAT mode, and it is exactly why you must cross-reference the
NSX NAT table (Section 5.1) to see the egress
identity. Node EXTERNAL-IP is also - for every
node, confirming all north-south reachability depends on NSX
routing/NAT, not on any externally assigned node address.
Using the data, an application pod reaching the external DNS server translates as follows:
assembly pod 192.168.148.2 (on worker node …wqxpw)
-> guest CNI masquerade -> node 10.55.18.4 (Supervisor Namespace Network)
-> NSX Tier-1 SNAT ------> egress IP 10.88.32.x (from NSX, NOT visible in k8s)
-> NSX Tier-0 -----------> 192.168.204.1 ------> external DNS 192.168.200.7
External DNS sees: 10.88.32.x (never 192.168.148.2, never 10.55.18.4)
This single trace ties together all four captures: the pod/node IPs come from this report, the SNAT egress IP from the NAT table, and the upstream path from the routing table.
k8s-ip-report-20260615-170232-nat.txt (NAT
enabled)================================================================================
1. NODE IPs
================================================================================
NODE INTERNAL-IP EXTERNAL-IP HOSTNAME
dev-cluster-01-dev-cluster-01-np-01-cmrnj-8bzc7-jxb7t 10.55.18.5 -
dev-cluster-01-dev-cluster-01-np-01-cmrnj-8bzc7-wqxpw 10.55.18.4 -
dev-cluster-01-dev-cluster-01-np-01-cmrnj-8bzc7-zqvrl 10.55.18.3 -
dev-cluster-01-dpwx4-4bmzq 10.55.18.6 -
dev-cluster-01-dpwx4-x47ng 10.55.18.7 -
dev-cluster-01-dpwx4-zmwvt 10.55.18.2 -
================================================================================
2. NODE POD CIDRs
================================================================================
NODE INTERNAL-IP POD-CIDR POD-CIDRS
dev-cluster-01-dev-cluster-01-np-01-cmrnj-8bzc7-jxb7t 10.55.18.5 192.168.145.0/24 192.168.145.0/24
dev-cluster-01-dev-cluster-01-np-01-cmrnj-8bzc7-wqxpw 10.55.18.4 192.168.148.0/24 192.168.148.0/24
dev-cluster-01-dev-cluster-01-np-01-cmrnj-8bzc7-zqvrl 10.55.18.3 192.168.147.0/24 192.168.147.0/24
dev-cluster-01-dpwx4-4bmzq 10.55.18.6 192.168.149.0/24 192.168.149.0/24
dev-cluster-01-dpwx4-x47ng 10.55.18.7 192.168.150.0/24 192.168.150.0/24
dev-cluster-01-dpwx4-zmwvt 10.55.18.2 192.168.144.0/24 192.168.144.0/24
================================================================================
3. CLUSTER SERVICE CIDR
================================================================================
SOURCE SERVICE-CIDR
ServiceCIDR/kubernetes 10.96.0.0/12
================================================================================
4. NODE POD CIDRs AND CLUSTER SERVICE CIDR
================================================================================
NODE INTERNAL-IP NODE-POD-CIDR CLUSTER-SERVICE-CIDR
dev-cluster-01-dev-cluster-01-np-01-cmrnj-8bzc7-jxb7t 10.55.18.5 192.168.145.0/24 10.96.0.0/12
dev-cluster-01-dev-cluster-01-np-01-cmrnj-8bzc7-wqxpw 10.55.18.4 192.168.148.0/24 10.96.0.0/12
dev-cluster-01-dev-cluster-01-np-01-cmrnj-8bzc7-zqvrl 10.55.18.3 192.168.147.0/24 10.96.0.0/12
dev-cluster-01-dpwx4-4bmzq 10.55.18.6 192.168.149.0/24 10.96.0.0/12
dev-cluster-01-dpwx4-x47ng 10.55.18.7 192.168.150.0/24 10.96.0.0/12
dev-cluster-01-dpwx4-zmwvt 10.55.18.2 192.168.144.0/24 10.96.0.0/12
================================================================================
5. POD IPs
================================================================================
NAMESPACE POD POD-IP NODE
assembly-lb assembly-deployment-7d6f774bd5-gj2dx 192.168.148.2 dev-cluster-01-dev-cluster-01-np-01-cmrnj-8bzc7-wqxpw
assembly-lb assembly-deployment-7d6f774bd5-rwpvs 192.168.145.2 dev-cluster-01-dev-cluster-01-np-01-cmrnj-8bzc7-jxb7t
assembly-lb assembly-deployment-7d6f774bd5-xt4ws 192.168.147.2 dev-cluster-01-dev-cluster-01-np-01-cmrnj-8bzc7-zqvrl
kube-system antrea-agent-dl2xn 10.55.18.3 dev-cluster-01-dev-cluster-01-np-01-cmrnj-8bzc7-zqvrl
kube-system antrea-agent-fm9f4 10.55.18.2 dev-cluster-01-dpwx4-zmwvt
kube-system antrea-agent-l6vzz 10.55.18.6 dev-cluster-01-dpwx4-4bmzq
kube-system antrea-agent-pz7lt 10.55.18.4 dev-cluster-01-dev-cluster-01-np-01-cmrnj-8bzc7-wqxpw
kube-system antrea-agent-rkfzb 10.55.18.7 dev-cluster-01-dpwx4-x47ng
kube-system antrea-agent-xdgnj 10.55.18.5 dev-cluster-01-dev-cluster-01-np-01-cmrnj-8bzc7-jxb7t
kube-system antrea-controller-5cb948c4d9-b48r6 10.55.18.2 dev-cluster-01-dpwx4-zmwvt
kube-system coredns-59ccbb77cb-dmmn2 192.168.144.7 dev-cluster-01-dpwx4-zmwvt
kube-system coredns-59ccbb77cb-qmknx 192.168.144.3 dev-cluster-01-dpwx4-zmwvt
kube-system etcd-dev-cluster-01-dpwx4-4bmzq 10.55.18.6 dev-cluster-01-dpwx4-4bmzq
kube-system etcd-dev-cluster-01-dpwx4-x47ng 10.55.18.7 dev-cluster-01-dpwx4-x47ng
kube-system etcd-dev-cluster-01-dpwx4-zmwvt 10.55.18.2 dev-cluster-01-dpwx4-zmwvt
kube-system kube-apiserver-dev-cluster-01-dpwx4-4bmzq 10.55.18.6 dev-cluster-01-dpwx4-4bmzq
kube-system kube-apiserver-dev-cluster-01-dpwx4-x47ng 10.55.18.7 dev-cluster-01-dpwx4-x47ng
kube-system kube-apiserver-dev-cluster-01-dpwx4-zmwvt 10.55.18.2 dev-cluster-01-dpwx4-zmwvt
kube-system kube-controller-manager-dev-cluster-01-dpwx4-4bmzq 10.55.18.6 dev-cluster-01-dpwx4-4bmzq
kube-system kube-controller-manager-dev-cluster-01-dpwx4-x47ng 10.55.18.7 dev-cluster-01-dpwx4-x47ng
kube-system kube-controller-manager-dev-cluster-01-dpwx4-zmwvt 10.55.18.2 dev-cluster-01-dpwx4-zmwvt
kube-system kube-proxy-5rd6n 10.55.18.3 dev-cluster-01-dev-cluster-01-np-01-cmrnj-8bzc7-zqvrl
kube-system kube-proxy-6t696 10.55.18.2 dev-cluster-01-dpwx4-zmwvt
kube-system kube-proxy-bxhlp 10.55.18.4 dev-cluster-01-dev-cluster-01-np-01-cmrnj-8bzc7-wqxpw
kube-system kube-proxy-gwqk2 10.55.18.6 dev-cluster-01-dpwx4-4bmzq
kube-system kube-proxy-jbj5p 10.55.18.7 dev-cluster-01-dpwx4-x47ng
kube-system kube-proxy-lzrvl 10.55.18.5 dev-cluster-01-dev-cluster-01-np-01-cmrnj-8bzc7-jxb7t
kube-system kube-scheduler-dev-cluster-01-dpwx4-4bmzq 10.55.18.6 dev-cluster-01-dpwx4-4bmzq
kube-system kube-scheduler-dev-cluster-01-dpwx4-x47ng 10.55.18.7 dev-cluster-01-dpwx4-x47ng
kube-system kube-scheduler-dev-cluster-01-dpwx4-zmwvt 10.55.18.2 dev-cluster-01-dpwx4-zmwvt
kube-system metrics-server-58cf45ddb5-b4ltz 192.168.144.5 dev-cluster-01-dpwx4-zmwvt
kube-system snapshot-controller-7cdcbf58d7-8rrq7 192.168.144.6 dev-cluster-01-dpwx4-zmwvt
network-test ol9-network-test-f556cf6b6-4vz2s 192.168.147.3 dev-cluster-01-dev-cluster-01-np-01-cmrnj-8bzc7-zqvrl
secretgen-controller secretgen-controller-6bcb99fd4b-fkklg 192.168.144.2 dev-cluster-01-dpwx4-zmwvt
tkg-system kapp-controller-86cdc46947-6dtdn 10.55.18.2 dev-cluster-01-dpwx4-zmwvt
vmware-system-auth guest-cluster-auth-svc-fcz5n 10.55.18.2 dev-cluster-01-dpwx4-zmwvt
vmware-system-auth guest-cluster-auth-svc-g5xz4 10.55.18.7 dev-cluster-01-dpwx4-x47ng
vmware-system-auth guest-cluster-auth-svc-xtbkt 10.55.18.6 dev-cluster-01-dpwx4-4bmzq
vmware-system-cloud-provider guest-cluster-cloud-provider-69bcc5b6d4-hbpjv 10.55.18.2 dev-cluster-01-dpwx4-zmwvt
vmware-system-csi vsphere-csi-controller-9949895b8-q6z8l 192.168.144.4 dev-cluster-01-dpwx4-zmwvt
vmware-system-csi vsphere-csi-node-2hqtn 10.55.18.6 dev-cluster-01-dpwx4-4bmzq
vmware-system-csi vsphere-csi-node-6g5p8 10.55.18.2 dev-cluster-01-dpwx4-zmwvt
vmware-system-csi vsphere-csi-node-jndf7 10.55.18.5 dev-cluster-01-dev-cluster-01-np-01-cmrnj-8bzc7-jxb7t
vmware-system-csi vsphere-csi-node-lmmh9 10.55.18.7 dev-cluster-01-dpwx4-x47ng
vmware-system-csi vsphere-csi-node-nc6rr 10.55.18.3 dev-cluster-01-dev-cluster-01-np-01-cmrnj-8bzc7-zqvrl
vmware-system-csi vsphere-csi-node-trwxt 10.55.18.4 dev-cluster-01-dev-cluster-01-np-01-cmrnj-8bzc7-wqxpw
================================================================================
6. SERVICE CLUSTER IPs
================================================================================
NAMESPACE SERVICE TYPE CLUSTER-IP PORTS
assembly-lb assembly-service LoadBalancer 10.105.77.235 8080/TCP
default kubernetes ClusterIP 10.96.0.1 443/TCP
default supervisor ClusterIP None 6443/TCP
kube-system antrea ClusterIP 10.97.123.99 443/TCP
kube-system kube-dns ClusterIP 10.96.0.10 53/UDP 53/TCP 9153/TCP
kube-system metrics-server ClusterIP 10.98.253.184 443/TCP
network-test ol9-network-test-lb LoadBalancer 10.104.102.169 22/TCP
tkg-system packaging-api ClusterIP 10.102.6.58 443/TCP 8080/TCP
vmware-system-csi vsphere-csi-controller ClusterIP 10.110.237.62 2112/TCP 2113/TCP
================================================================================
7. LOADBALANCER SERVICE EXTERNAL IPs / HOSTNAMES
================================================================================
NAMESPACE SERVICE TYPE CLUSTER-IP EXTERNAL-IP/HOSTNAME PORTS
assembly-lb assembly-service LoadBalancer 10.105.77.235 10.77.64.3 8080/TCP
network-test ol9-network-test-lb LoadBalancer 10.104.102.169 10.77.64.4 22/TCP
================================================================================
8. SERVICE externalIPs FIELD
================================================================================
NAMESPACE SERVICE TYPE EXTERNAL-IPS
================================================================================
9. ENDPOINT IPs
================================================================================
NAMESPACE SERVICE ENDPOINT-IPS
assembly-lb assembly-service 192.168.145.2 192.168.147.2 192.168.148.2
default kubernetes 10.55.18.2 10.55.18.6 10.55.18.7
default supervisor 10.77.68.1
kube-system antrea 10.55.18.2
kube-system kube-dns 192.168.144.3 192.168.144.7
kube-system metrics-server 192.168.144.5
network-test ol9-network-test-lb 192.168.147.3
tkg-system packaging-api 10.55.18.2
vmware-system-csi vsphere-csi-controller 192.168.144.4