Skip to main content

AKS Guide

This guide covers everything specific to running k8s-multitenant on Azure Kubernetes Service (AKS).

Prerequisites​

1. NetworkPolicy enforcement​

NetworkPolicy is only enforced if the cluster was created with a supported network policy engine:

OptionHow to enableNotes
Azure CNI + Azure Network Policyaz aks create --network-plugin azure --network-policy azureRequires Azure CNI (not kubenet)
Azure CNI + Ciliumaz aks create --network-plugin azure --network-plugin-mode overlay --network-policy ciliumRecommended for new clusters
kubenet + Calicoaz aks create --network-plugin kubenet --network-policy calicoLegacy option
warning

You cannot add network policy to an existing cluster. It must be specified at creation time.

2. RBAC — Azure AD groups as Kubernetes subjects​

Enable AKS-managed Azure AD integration so Azure AD group Object IDs map to Kubernetes groups:

az aks create \
  --name my-cluster \
  --resource-group my-rg \
  --enable-aad \
  --aad-admin-group-object-ids <admin-group-object-id>

For existing clusters:

az aks update \
  --name my-cluster \
  --resource-group my-rg \
  --enable-aad \
  --aad-admin-group-object-ids <admin-group-object-id>

Once enabled, use the Azure AD Group Object ID (a UUID) as the subject name:

rbac:
  subjects:
    - kind: Group
      name: "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"   # Azure AD Group Object ID
      apiGroup: rbac.authorization.k8s.io

Find the Object ID:

az ad group show --group "team-alpha-admins" --query id --output tsv

Finding your vpcCidr​

# Get the address space of your AKS VNet
az network vnet list \
  --query "[?contains(name, 'aks')].{name:name, cidr:addressSpace.addressPrefixes[0]}" \
  --output table
# Example output: 10.0.0.0/8

Set networkPolicy.vpcCidr to this value. This allows:

  • Azure Load Balancer health probe traffic into pods (168.63.129.16/32 — add separately if strict mode needed)
  • Egress to Azure Database for PostgreSQL Flexible Server, Azure Cache for Redis, and other VNet-peered services

Example values​

global:
  labels:
    managed-by: platform-team
    cloud: azure

tools:
  create: true
  namespace: k8s-tools

tenants:
  - name: team-alpha
    labels:
      environment: production
      cost-center: eng-platform
    rbac:
      subjects:
        # Azure AD Group Object ID
        - kind: Group
          name: "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
          apiGroup: rbac.authorization.k8s.io
    networkPolicy:
      enabled: true

  - name: team-beta
    labels:
      environment: production
      cost-center: eng-product
    rbac:
      subjects:
        - kind: Group
          name: "yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy"
          apiGroup: rbac.authorization.k8s.io
    networkPolicy:
      enabled: true

rbac:
  create: true
  serviceAccountName: default

resourceQuota:
  enabled: true
  default:
    requests.cpu: "2"
    requests.memory: 4Gi
    limits.cpu: "4"
    limits.memory: 8Gi
    pods: "20"
    services: "5"

networkPolicy:
  enabled: true
  vpcCidr: "10.0.0.0/8"     # your AKS VNet address space
  allowInternetEgress: false

The full example is at examples/aks/values.yaml.

Workload Identity​

For pod-level access to Azure resources (Key Vault, Storage, Service Bus), use AKS Workload Identity:

# Enable on the cluster
az aks update --name my-cluster --resource-group my-rg --enable-oidc-issuer --enable-workload-identity

Then annotate the pod's ServiceAccount:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: my-app
  namespace: team-alpha
  annotations:
    azure.workload.identity/client-id: <managed-identity-client-id>

Troubleshooting on AKS​

NetworkPolicy has no effect

# Confirm network policy is set to 'azure' or 'cilium'
az aks show --name my-cluster --resource-group my-rg \
  --query networkProfile.networkPolicy --output tsv

Azure AD group binding not working

# Confirm AAD integration is enabled
az aks show --name my-cluster --resource-group my-rg \
  --query aadProfile --output json

Ensure the user is a member of the group and try re-fetching credentials:

az aks get-credentials --name my-cluster --resource-group my-rg --overwrite-existing

Pods can't reach Azure Database for PostgreSQL Verify networkPolicy.vpcCidr covers the VNet address space used by the PostgreSQL private endpoint.