Dynamic Kubernetes Persistent Volume Provisioning with NetApp Trident and Cloud Volumes ONTAP

If you’re operating in Kubernetes or thinking about it, you know there are a lot of benefits to scaling containers across a cluster. But no matter how much leverage cluster deployment gives you, you still need to find somewhere to store all of your data. How exactly does storage work in Kubernetes?

Let’s start with what we do know: Kubernetes provides an extensible framework for managing data storage within a Kubernetes cluster. At a high level, there are the two main options available for Kubernetes persistent volume provisioning, namely static and dynamic persistent volume allocation. When using static allocation, administrators need to pre-allocate all persistent volumes. That requires a precise foreknowledge of how the cluster’s storage resources will be used.

The other option is to use dynamic provisioning. With dynamic provisioning, persistent volumes are created more organically, giving the cluster an advantage when it comes to dealing with changes to storage requirements. Don’t know how much storage your pods are going to take up? Better go with dynamic provisioning. Most of the time in Kubernetes deployments, this will be the case.

In this article, we’ll take a closer look at how Kubernetes can make use of persistent volumes that are dynamically allocated in Cloud Volumes ONTAP with the help of the NetApp Trident provisioner, and all the great benefits this provides in terms of cost effectiveness, flexibility, and centralized storage management.

Provisioning Kubernetes Volumes

Persistent volumes provide the means for data storage to exist independently of pods and containers. Each pod specification may include a number of persistent volume claims, which instruct the cluster to search for a suitable persistent volume based on the requested size, access mode, labels, etc. This gives persistent volumes the additional advantage of separating out the request for storage by application developers deploying a pod, from the low-level details of how Kubernetes administrators decide to make the storage available to the cluster.

How does dynamic provisioning help?

Using dynamic provisioning in a Kubernetes cluster reduces a lot of the administrative overhead involved in manually creating persistent volumes, both as and when they are required. Automatically allocating and deallocating persistent volumes in response to persistent volume claims can also help to reduce wasted storage that is allocated but never used. Persistent volume claims that are fulfilled dynamically make use of a storage class attribute to specify the type of storage required, which allows for data to be tiered by I/O performance profile.

trident storage class definition
Kubernetes comes with a variety of internal provisioners that dynamically allocate storage using Amazon EBS, Google Persistent Disk, Azure Disk Storage, and many other storage platforms. However, that’s where the support for storage ends: storage management features, such as creating and restoring backups, ensuring high availability, and provisioning test copies of persistent volumes are all out of scope for Kubernetes and must be catered for by cluster administrators. In a hybrid or multicloud environment, this can lead to a multitude of different solutions that will increase the complexity of managing storage across the enterprise. What can you do to avoid that?

Dynamic Provisioning with Cloud Volumes ONTAP and NetApp Trident

Cloud Volumes ONTAP snapshots are created instantlyNetApp Trident is a dynamic Kubernetes provisioner that uses Cloud Volumes ONTAP, leveraging Azure storage or AWS storage, or an on-premises ONTAP appliance as the storage back end for Kubernetes persistent volumes. This enables Kubernetes clusters to take advantage of the power of NetApp storage management for persistent volume provisioning in on-premises, hybrid cloud, or multicloud environments.

Cloud Volumes ONTAP uses native cloud compute and storage services to create a virtual NetApp storage appliance in either AWS or Azure. This provides many features and benefits that just don’t exist when directly allocating storage, including:

  • Thin provisioning: Large persistent volumes can be allocated using Cloud Volumes ONTAP without actually consuming storage space. Instead, volumes grow automatically as required, which avoids the wasteful allocation of cloud storage based on initial estimated requirements that may end up not being used.
  • Deduplication and compression: Cloud Volumes ONTAP helps to reduce cloud storage footprint by using built-in technologies for transparently deduplicating and compressing the data that it stores. Both of these features can be used together to cut down storage overhead by as much as 50-70% in some cases, which leads to significant operational cost savings.
  • Storage tiering: Kubernetes administrators can easily map different storage classes to data aggregates in Cloud Volumes ONTAP that are built from any of the available Amazon EBS disk types or from Standard and Premium Azure Disks. Cloud Volumes ONTAP can also automatically tier the cold data in persistent volumes to low cost object storage, such as Amazon S3 or Azure Blob.
  • ConclusionPersistent volume cloning: Using NetApp FlexClone®, Cloud Volumes ONTAP can instantly create writable clones of an existing persistent volume of any size. Trident can integrate with FlexClone to deploy clones of an existing persistent volume instead of always allocating new storage. This has huge advantages for setting up test clusters and DevOps CI/CD pipelines, and for cutting down on storage usage and costs.
  • Instant backup and restore: ONTAP snapshots are always created instantly, regardless of source data size, are highly space efficient, and can also be instantly restored back to the source volume. Using NetApp SnapCenter®, application-aware snapshots are created in conjunction with running applications to ensure that in-flight I/O operations are handled appropriately.
  • High availability: Native cloud storage solutions, such as Amazon EBS and Azure Disk, provide local redundancy within an Availability Zone. Cloud Volumes ONTAP HA offers greater levels of protection by setting up a pair of active-active storage nodes across Availability Zones that synchronously replicate changes to each other.
  • Data replication: SnapMirror® is NetApp’s solution for incremental, block-level replication of storage volumes allocated using ONTAP storage services. This allows persistent volumes to be replicated to and from the cloud and on-premises systems or between different deployments of Cloud Volumes ONTAP.
  • Tooling: Trident provides the tridentctl command line tool for administrators to easily review and manage the storage resources they are using. Cloud Manager is the web-based GUI interface used for deploying and managing Cloud Volumes ONTAP. Both Trident and Cloud Manager also provide RESTful APIs to enable process automation.
  • Hybrid/Multicloud support: Trident supports on-premises ONTAP storage appliances, as well as deployments of Cloud Volumes ONTAP in both AWS and Azure. This gives cluster administrators a uniform interface for managing their storage wherever it resides, and delivers the advantages of NetApp storage management to all of their environments.

For all the good that Kubernetes does, you still need to allocate storage for the workload you’re running on it. Dynamic storage allocation for Kubernetes persistent volumes simplifies the deployment of persistent volumes within Kubernetes clusters, doing away with the need for administrators to manually allocate storage in advance. NetApp Trident enables persistent volumes to be provisioned using Cloud Volumes ONTAP, which perfectly complements the container orchestration capabilities of Kubernetes with NetApp’s enterprise-grade storage solutions.

New call-to-action