Azure AKS Networking: VNET Integration and CNI
AKS gives you real choices about how pods get IPs, how traffic leaves the cluster, and whether the API server is public — and the defaults aren't always what production wants. This guide covers the models, the commands, and the failure modes we've hit running AKS in anger.
Table of Contents
- AKS Networking Models
- Azure CNI vs Kubenet
- VNET Integration
- Egress: how traffic leaves the cluster
- Private AKS Clusters
- Network Policy
- DNS and Ingress
- Common Issues
AKS Networking Models
Two models determine how pods get IP addresses:
| Kubenet | Azure CNI | |
|---|---|---|
| Pod IPs | From an overlay network | Directly from the VNET subnet |
| Routable | Only within cluster | Routable on the VNET |
| Scale | IP-efficient | Requires larger subnet |
| Use case | Small clusters, IP-constrained | Production, hybrid networking |
The question that decides it: do pods need to be addressable from outside the cluster — by on-prem networks over ExpressRoute, by VNET-peered services, by NSG rules? If yes, Azure CNI. If you're conserving IPs in a crowded VNET, Kubenet or CNI Overlay.
Azure CNI vs Kubenet
Kubenet (Default)
az aks create \
--resource-group my-rg \
--name my-cluster \
--network-plugin kubenet \
--pod-cidr 10.244.0.0/16
Pods get IPs from 10.244.0.0/16 — an overlay that doesn't exist in your VNET. Traffic NATs through the node IP to leave the cluster, so VNET services see node IPs, not pod IPs. Cheap on addresses, awkward for anything that needs to reach pods directly.
Azure CNI
az aks create \
--resource-group my-rg \
--name my-cluster \
--network-plugin azure \
--vnet-subnet-id /subscriptions/.../subnets/aks-subnet \
--pod-subnet-id /subscriptions/.../subnets/pod-subnet # optional: separate pod subnet
Pods get IPs directly from the VNET subnet — first-class citizens that NSGs, route tables, and on-prem firewalls can all see. The cost is real IPs: a 30-pod node consumes 30 subnet addresses whether the pods are busy or not.
Azure CNI Overlay (Best of Both)
az aks create \
--resource-group my-rg \
--name my-cluster \
--network-plugin azure \
--network-plugin-mode overlay
Pods get IPs from a private overlay (like Kubenet — doesn't consume VNET addresses) but egress traffic appears as node IPs on the VNET. For most new clusters this is the right default: CNI features without the IP math. Also required if you want Cilium as the dataplane, which is where the ecosystem is heading.
VNET Integration
Custom VNET
Letting AKS create its own VNET works for demos; production clusters belong in a VNET you designed — peered, firewalled, and addressed to coexist with everything else:
# Create VNET first
az network vnet create \
--resource-group my-rg \
--name aks-vnet \
--address-prefix 10.0.0.0/8 \
--subnet-name aks-subnet \
--subnet-prefix 10.240.0.0/16
az aks create \
--resource-group my-rg \
--name my-cluster \
--vnet-subnet-id $(az network vnet subnet show -g my-rg --vnet-name aks-vnet -n aks-subnet --query id -o tsv) \
--network-plugin azure \
--service-cidr 10.0.0.0/16 \
--dns-service-ip 10.0.0.10
--service-cidr and --dns-service-ip are the two things people forget: they define the internal ClusterIP range and where CoreDNS listens inside it. Pick a range that doesn't collide with the VNET or any peered/on-prem network — the collision only surfaces when a pod tries to reach an overlapping address and mysteriously fails.
Subnet Sizing
Azure CNI assigns a real IP per pod. Plan for:
Subnet size = (max pods per node × max nodes) + system pods + headroom
Example: 50 nodes × 30 pods = 1500 IPs → /20 (4094 usable) is safe. Azure also reserves 5 addresses per subnet. Running out isn't a graceful failure — new pods just pend, and the fix (a bigger subnet or a new node pool's subnet) is a rebuild-level change, so size generously up front.
Egress: how traffic leaves the cluster
Outbound SNAT defaults to the load balancer AKS creates — fine until you hit SNAT port exhaustion on chatty workloads (thousands of short outbound connections). Two fixes:
- NAT Gateway (
--nat-gateway-managedor bring your own) — dedicated SNAT ports per IP, the standard answer to exhaustion. - Outbound UDRs — route egress through an Azure Firewall or NVA for inspection and fixed egress IPs that partners can whitelist.
If a third party needs your source IP for an allowlist, NAT Gateway with a dedicated public IP is the clean answer — the default LB egress IP can change.
Private AKS Clusters
az aks create \
--resource-group my-rg \
--name private-cluster \
--enable-private-cluster \
--private-dns-zone system
The API server gets a private IP reachable only inside the VNET (or over VPN/ExpressRoute/peering) — nothing public to attack. The trade-off is operational: kubectl from your laptop no longer works without network plumbing. Access options:
az aks command invoke— one-off commands through the Azure control plane, great for break-glass.- VPN/ExpressRoute + the private DNS zone for real
kubectl. - Azure Bastion + a jumpbox VM in a peered VNET.
--private-dns-zone system lets AKS manage the privatelink.<region>.azmk8s.io zone itself; bring your own zone if DNS is centrally managed.
Network Policy
# Enable at cluster creation — cannot be toggled later
az aks create \
--resource-group my-rg \
--name my-cluster \
--network-plugin azure \
--network-policy azure # or calico / cilium (overlay only)
Then standard Kubernetes NetworkPolicies apply:
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: deny-all
spec:
podSelector: {}
policyTypes:
- Ingress
- Egress
The pattern that works: deny-all per namespace as the baseline, then explicit allows (DNS egress to kube-system, ingress from the ingress namespace, egress to the database). Without a policy engine, every pod can reach every pod — one compromised container becomes a flat network's worth of lateral movement.
DNS and Ingress
Azure DNS for Private Clusters
az aks create \
--enable-private-cluster \
--private-dns-zone /subscriptions/.../privatednszones/privatelink.uksouth.azmk8s.io
Application Gateway Ingress Controller
az aks create \
--resource-group my-rg \
--name my-cluster \
--enable-addon ingress-appgw \
--appgw-name my-appgw \
--appgw-subnet-id /subscriptions/.../subnets/appgw-subnet
AGIC turns a managed Application Gateway (L7, WAF-capable) into your ingress — the Azure-native answer when you want a WAF in front without running an in-cluster NGINX. For simpler needs, the NGINX ingress controller + a Standard LB is the cheaper path.
Internal Load Balancer
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/azure-load-balancer-internal: "true"
spec:
type: LoadBalancer
# ...
type: LoadBalancer provisions a public Azure LB by default — the azure-load-balancer-internal annotation is what keeps a Service private. Forgetting it is the classic way an internal API ends up internet-facing.
Common Issues
Pods Can't Resolve DNS
kubectl exec -it <pod> -- nslookup kubernetes.default
# Check CoreDNS logs:
kubectl logs -n kube-system -l k8s-app=kube-dns
Pods Can't Reach External IPs
Check Network Security Groups on the AKS subnet — outbound rules may be too restrictive — and whether a UDR is sending egress to a firewall that's dropping it.
CNI IP Exhaustion
# Check available IPs
az network vnet subnet show -g my-rg --vnet-name aks-vnet -n aks-subnet \
--query "availableIpAddressCount"
Near-zero means new pods can't get addresses — they pend with FailedCreatePodSandBox events naming CNI. The fix is more addresses: a bigger pod subnet, a new node pool on a different subnet, or CNI Overlay.
Private Cluster DNS Not Resolving
Ensure the private DNS zone is linked to the VNET where your VM/bastion/kubectl host lives — the zone existing isn't enough; the link is what makes resolution work.
Related Articles
Our tutorials and troubleshooting runbooks are written by a DevOps practitioner with hands-on experience operating Kubernetes, Terraform, CI/CD pipelines, and Linux infrastructure in production. Commands, manifests, and configurations are verified against current tooling releases. Learn more about our editorial process or report a correction or outdated flag.