Containers Data / Storage

Azure NetApp Files + Trident = Dynamic and Persistent Storage for Kubernetes

My eleven year old daughter recently asked me ‘the question’. You know the one…

“Hey dadda, what’s Kubernetes?”

How do you explain Kubernetes to an eleven year old?! Check out this video to learn how. Brilliant.

I am not a Kubernetes expert. I am learning, probably just like you are. Creating resources, tearing down resources, building them again, kubectl-ing, scratching my head, scratching my head some more, rinse and repeat. I can understand how powerful it is. I can also understand how complex it is.

As an engineer, I am constantly trying to distill complex things down to their simplest form. I figure out how to do something by reading documentation, blogs, dissecting code, and a humorous amount of trial and error. If I don’t distill this information and document it, I’ll have to do it all over again in a few months as my memory is just awful. I do enjoy this process, and I hope these types of posts can help you get to where you are going a tiny bit faster.

If you want to deploy or demonstrate dymamically provisioned, persistent storage without being a Kubernetes expert, this is the post for you.

Let’s start with some basic definitions:

  1. Kubernetes; container orchestration system
  2. Azure Kubernetes Service (AKS); a fully managed Kubernetes cluster service provided by Microsoft
  3. NetApp Trident; dynamic, persistent, storage orchestrator for Kubernetes (or Docker)
  4. Azure NetApp Files; lightning fast, enterprise grade, file storage service (NFS/SMB) provided by Microsoft

At the end of this tutorial you will have deployed AKS, installed NetApp Trident, configured a Trident backend to dynamically provision Azure NetApp Files volumes, and have a running nginx deployment being served by Azure NetApp Files storage.

Before we dive in… You will need a Linux operating system to interact with Trident (tridentctl). I am using an Ubuntu VM running in Azure. You could use the Windows Subsystem for Linux or any Linux distro on the supported host operating systems list found here.

Ok, let’s dive in!

Deploy your Kubernetes cluster using the Azure Kubernetes Service (AKS)

  1. From within the Azure portal, navigate to ‘Kubernetes services’ and click the ‘+Add’ button at the top, choose ‘Add Kubernetes cluster’.AKS Add Cluster
  2. You can accept the default settings for everything. You will need to provide the resource group and give your cluster a name. Feel free to reduce the number of nodes and node size to save yourself some money. A single node is enough for this demo. AKS will create all of the required Azure networking components for you.

Create a delegated subnet for Azure NetApp Files

  1. Navigate to ‘Virtual networks’ within the Azure portal. Find your newly created virtual network. It should have a prefix similiar to ‘aks-vnet’. Click on the name of the VNet.AKS VNet
  2. Click on ‘Subnets’ and select ‘+Subnet’ from the top toolbar.ANF New Subnet
  3. Give your subnet a name like ‘’ and under the ‘Subnet delegation’ heading, select ‘Microsoft.Netapp/volumes’. Do not change anything else. Click ‘OK’.ANF Subnet Detail

Create a new NetApp account within the Azure NetApp Files Service

  1. Navigate to the Azure NetApp Files service, click on the ‘+Add’ button.ANF Add Account
  2. Give your account a name, choose a resource group. IMPORTANT: For the location, choose the same region as your AKS cluster. Click the ‘Create’ button.ANF New Account

Create a Azure NetApp Files Capacity Pool

  1. Navigate to your NetApp account, click on ‘Capacity pools’, click ‘+Add pool’, give it a name, choose ‘Standard’ for the service level, leave the QoS type as Auto. Click ‘Create’.AKS Add Pool

Install ‘kubectl’, Helm, and the Azure CLI on your Ubuntu VM (or WSL)

  1. Install ‘kubectl’: sudo snap install kubectl --classic
  2. Install Helm: sudo snap install helm --classic
  3. Install Azure CLI: curl -sL | sudo bash
  4. Login to your Azure account using Azure CLI: az login

Connect to your Kubernetes cluster

  1. From within the Azure portal, navigate to your Kubernetes cluster and click the ‘Connect’ button, execute the ‘az account set…’ and ‘az aks get-credentials…’ commands from your Linux terminal.Connect AKS
  2. Verify you are connected to your Kubernetes cluster: kubectl get deployments --all-namespaces=true
  3. It should look something like this:Verify AKS

Install Trident with Helm

I encourage you to go read the official documentation here. I have summarized the commands below.

  1. Download the Trident installer bundle: wget
  2. Extract the bundle: tar -xf trident-installer-21.07.0.tar.gz
  3. Copy ‘tridentctl’ to a directory in your $PATH (i.e., ‘/usr/local/bin’): sudo cp trident-installer/tridentctl /usr/local/bin
  4. Change directory to the trident-installer Helm chart: cd trident-installer/helm
  5. Deploy Trident using Helm: helm install trident trident-operator-21.07.0.tgz --namespace trident --create-namespace
  6. It should look like this:

Clone my ‘ANF_Trident_AKS’ GitHub Repository

This repo has some boiler plate code that we will modify to demonstrate Trident in your AKS environment.

  1. From your Linux workstation, go back to your home directory: cd ~
  2. Clone the ‘ANF_Trident_AKS’ Repository and change directory to ‘ANF_Trident_AKS’:
    • git clone
    • cd ANF_Trident_AKS

Create an Azure Service Principal

  1. The service principal is how Trident communicates with Azure to manipulate your Azure NetApp Files resources: az ad sp create-for-rbac --name "http://netapptrident"
  2. The output should look like this:Create SPN

Create your Trident backend for Azure NetApp Files

  1. Using your preferred text editor, complete the following fields inside the ‘anf_backend.json’ file:
    • subscriptionID: your Azure Subscription ID
    • tenantID: your Azure Tenant ID (from the output of ‘az ad sp’ in the previous step)
    • clientID: your appID (from the output of ‘az ad sp’ in the previous step)
    • clientSecret: your ‘password’ (from the output of ‘az ad sp’ in the previous step)
    • location: the location of your Azure NetApp Files capacity pool
    • serviceLevel: the service level of your Azure NetApp Files capacity pool
    • virtualNetwork: the virtual network that was created by AKS
    • subnet: the Azure NetApp Files delegated subnet
  2. The file should look similar to this:
  3. Instruct Trident to create the ANF backend in the ‘trident’ namespace: tridentctl -n trident create backend -f anf_backend.json Create Backend

Create a Kubernetes storage class

  1. Instruct Kubernetes to create a storage class that will reference our Trident backend: kubectl apply -f anf_sc.yaml

Create a Kubernetes persistent volume claim (PVC)

  1. Instruct Kubernetes to create a persistent volume claim: kubectl apply -f anf_pvc.yaml
  2. Verify that Trident has dynamically provisioned an Azure NetApp Files volume. From within the Azure portal you should see that a new Azure NetApp Files volume has been created.ANF New Volume

Create a Kubernetes deployment that references our persistent volume claim (PVC)

  1. Instruct Kubernetes to deployment a simple nginx web server: kubectl apply -f nginx_deployment.yaml
  2. Instruct Kubernetes to give our nginx deployment a public IP address: kubectl expose deployment nginx-anf-trident --port=80 --type=LoadBalancerExpose nginxNotice that only after you ‘kubectl expose’ does our deployment get a public IP address as shown above in the ‘EXTERNAL-IP’ column.
  3. Get your nginx pod name (it should start with ‘nginx-anf-trident’): kubectl get podsGet Pod Name
  4. Assign your pod name to an environment variable to save some key strokes: pod=<paste your pod name from step 3 above here>
  5. Set permissions on our Azure NetApp Files volume. Our volume is nounted via NFSv3 to our pod’s ‘/usr/share/nginx/html’ directory. The permissions on this folder need to be ‘755’.
    • kubectl exec -it $pod -- chmod 755 /usr/share/nginx/html
  6. Copy our custom ‘index.html’ file to your pod’s ‘/usr/share/nginx/html’ directory: kubectl cp ./index.html $pod:/usr/share/nginx/html/
  7. Point your web browser to your deployment’s ‘EXTERNAL-IP’ from step 2. You should be greeted with our custom index.html.Welcome to ANF

Destroy our Kubernetes deployment and re-deploy

  1. Delete your Kubernetes service (external IP address) kubectl delete svc nginx-anf-trident
  2. Delete your Kubernetes nginx deployment kubectl delete -f nginx_deployment.yaml
  3. Instruct Kubernetes to deploy your nginx web server again kubectl apply -f nginx_deployment.yaml
  4. Instruct Kubernetes to expose your deployment, giving it a public IP address. kubectl expose deployment nginx-anf-trident --port=80 --type=LoadBalancer
  5. Get your new public IP address kubectl get svc
  6. Point your browser to the public IP address and notice that your custom index.html has persisted! Magic.

Cleaning Up

  1. Delete your Kubernetes service (external IP address) kubectl delete svc nginx-anf-trident
  2. Delete your Kubernetes nginx deployment kubectl delete -f nginx_deployment.yaml
  3. Delete your Azure NetApp Files persistent volume claim (PVC)WARNING: This command will instruct Trident to delete the Azure NetApp Files volume. kubectl delete -f anf_pvc.yaml
  4. Go back to the Azure portal and delete your Azure NetApp Files capacity pool.


Leave a Reply

%d bloggers like this: