More about Kubernetes Storage
- Azure Kubernetes Service: Configuring Persistent Volumes in AKS
- Kubernetes NFS: Two Quick Tutorials
- AWS Kubernetes Cluster: Quick Setup with EC2 and EKS
- Kubernetes Shared Storage: The Basics and a Quick Tutorial
- AWS ECS vs Kubernetes: An Unfair Comparison?
- Kubernetes Persistent Storage: Why, Where and How
- Kubernetes for Developers: A Deep Dive
- How to Provision Kubernetes Persistent Volumes for NFS Services
- Using Cloud Manager for Kubernetes Deployment
- Docker Volume Tutorial - Using Trident to Provision Storage
- How to Set Up MySQL Kubernetes Deployments
- Storage Efficiency for Improving Persistent Volume Storage Costs
- Protection for Persistent Data Storage in Kubernetes
- Dynamic Kubernetes Persistent Volume Provisioning
- Managing Stateful Applications in Kubernetes
- Understanding Kubernetes Persistent Volume Provisioning
- An Introduction to Kubernetes
Kubernetes persistent volumes greatly simplify the process of provisioning storage within a Kubernetes cluster, but with NetApp Trident and Cloud Volumes ONTAP you can instantly create highly-space-efficient, writable clones of existing persistent volumes in direct response to persistent volume claims.
In this article, we examine the way in which NetApp Trident facilitates persistent volume cloning, and how this is delivered by Cloud Volumes ONTAP using NetApp FlexClone® technology.
Kubernetes Persistent Volumes Data Cloning
Creating test copies of production data is very often a requirement for DevOps engineers implementing CI/CD pipelines or setting up staging clusters for pre-production testing. Automated software test suites will often mutate the data that they access, which means that a fresh copy of the data is required in order to repeat the testing, with test cycles sometimes executed hundreds of times. Ensuring faster TTM (Time To Market) and the delivery of high quality software relies upon developers working in parallel and executing tests as often as required.
Restoring data backups is not a scalable solution for creating test data sets: with the size of the source data involved it just takes too much time to perform a restore. This leads to reduced developer productivity when multiple, up-to-date copies of persistent volumes are required, which also have to be refreshed frequently. Using this approach also means that the amount of storage used to develop and support an application will be many times greater than the storage requirements of the production environment, causing a significant increase in cloud storage costs. That’s not something any developer wants to get blamed for.
The time taken to perform a restore and the storage space it uses can both be very wasteful, as usually most of the restored data remains unaffected by the software testing and must be recreated simply to bring the data to a consistent point or to get an up-to-date copy. In contrast, not only is NetApp FlexClone able to instantly clone a source volume of any size, it also does so with zero storage penalty for any storage size required. NetApp Trident helps you take advantage of this for persistent volumes in a Kubernetes cluster.
Cloning Kubernetes Persistent Volumes with NetApp Trident
NetApp Trident is a provisioner for Kubernetes that performs dynamic storage provisioning using Cloud Volumes ONTAP for AWS or Azure, or other on-premises ONTAP storage systems. The major advantage of using Trident is that containerized applications can benefit from NetApp’s enterprise data management features. One of these features is data cloning which can help speed up TTM and reduce storage footprint and costs.
This is how Kubernetes users can benefit: instead of always provisioning new persistent volumes in response to persistent volume claims, using Trident the user can clone existing volumes. To instruct Trident to create a clone, simply give it the name of the persistent volume claim that was used to provision the original persistent volume, which must also have been allocated through Trident. This information is provided through a metadata annotation on the persistent volume claim that is requesting the volume clone.
In the background, Trident will find the ONTAP volume that was used to provision storage for the original persistent volume claim and then use NetApp FlexClone to perform a data cloning operation. Clones are created by using a snapshot on the parent volume. Both the clone and parent volumes are free to accept changes to their data, which are written to new blocks using a redirect-on-write mechanism that protects blocks locked by a snapshot from being overwritten. Using this technique, the clone requires negligible storage space when it is first created, and only grows with the changes that are made to it.
By using the default reclaim policy of delete, the clone will automatically be deleted when the pod using it is destroyed. This can be used to conveniently clean up any allocated storage at the end of a round of testing, for example. Trident also supports an optional annotation to split the clone from the parent volume it is associated with. This turns the clone into an independent copy of the data, which means the space efficiencies discussed previously will be lost, however, this can be more suitable in scenarios when the clone will undergo a lot of changes, or when the state of a volume is needed to provide seed data for a new requirement.
Kubernetes persistent volumes are used to provide storage for pods in a Kubernetes cluster, however, the provisioners used to create them do not typically provide mechanisms to stand up copies of existing volumes for testing. NetApp Trident is able to achieve this through a simple set of annotations on a persistent volume claim, which leverage the capabilities of Cloud Volumes ONTAP to instantly provision writable clones of existing persistent volumes. Data cloning is just one of Cloud Volumes ONTAP’s cost-cutting features.