More about Kubernetes Storage
- Azure Kubernetes Service: Configuring Persistent Volumes in AKS
- Kubernetes NFS: Two Quick Tutorials
- AWS Kubernetes Cluster: Quick Setup with EC2 and EKS
- Kubernetes Shared Storage: The Basics and a Quick Tutorial
- AWS ECS vs Kubernetes: An Unfair Comparison?
- Kubernetes Persistent Storage: Why, Where and How
- Kubernetes for Developers: A Deep Dive
- How to Provision Kubernetes Persistent Volumes for NFS Services
- Using Cloud Manager for Kubernetes Deployment
- Docker Volume Tutorial - Using Trident to Provision Storage
- How to Set Up MySQL Kubernetes Deployments
- Kubernetes Persistent Volumes Cloning
- Storage Efficiency for Improving Persistent Volume Storage Costs
- Protection for Persistent Data Storage in Kubernetes
- Dynamic Kubernetes Persistent Volume Provisioning
- Understanding Kubernetes Persistent Volume Provisioning
- An Introduction to Kubernetes
Why the need for extra guarantees when it comes to stateful vs stateless? Whereas the storage used by stateless containerized applications can simply be re-initialized when a pod is rescheduled to different node in the cluster, stateful applications are recording business-critical information that must be preserved at all costs—that requires persistent storage with an independent lifetime.
In Kubernetes, persistent volumes fill that role, and can be created using a variety of storage provisioners. The actual back-end storage services each provisioner interfaces with is responsible for providing adequate data protection, such as backup/restore and high availability, and other data management features.
With data protection, high availability, and flexible provisioning across the cluster all outside the scope of Kubernetes, what can you do to help ensure those things for your Kubernetes deployments?
In this article, we will look at the options for deploying stateful applications using Kubernetes and discuss the ways in which Cloud Volumes ONTAP and NetApp Trident make it easier to manage, scale, and protect the data storage those applications depend upon.
Stateful Applications in Kubernetes
Kubernetes deployment persistent volumes are used to create a layer of abstraction between pods and their provisioned storage, allowing each to be managed separately. Pods use a persistent volume claim as a request for storage that is matched to a persistent volume by the Kubernetes cluster.
Using a reclaim policy of “retain”, a persistent volume will not be deleted or cleared down after the persistent volume claim is released. Because of this, in the event of a failure or the rescheduling of the pod onto a different node, the relationship between pod and storage can be re-established. Take a look at this example use case of deploying MySQL with WordPress where persistent volumes were used in this way to deploy a database server.
The manual creation of persistent volumes ahead of time is known as static provisioning and requires much more administrative overhead than dynamic provisioning. In dynamic provisioning, the cluster can automatically deploy storage in response to the persistent volume claims it receives. Stateful applications can take advantage of dynamic provisioning by using stateful sets, which use a volume claim template to allocate storage and then permanently bind the resulting persistent volume to the requesting pod. This is more useful for building a cluster of stateful pods, such as for NoSQL database systems like Cassandra.
In both static and dynamic provisioning scenarios, stateful applications have a very real need for reliable storage that is resilient against failure, and data protection features, such as backup and restore. In and of itself, these facilities are not provided by Kubernetes, but are instead delegated to the provisioner used and the storage backend. Users have to find outside solutions, such as Trident and Cloud Volumes ONTAP.
Provisioning Kubernetes Storage with NetApp Trident and Cloud Volumes ONTAP
NetApp Trident is a dynamic storage provisioner for Kubernetes that fulfils persistent volume claims using storage managed by Cloud Volumes ONTAP as well as other on-prem NetApp storage systems. By using cloud-native compute and storage infrastructure services, Cloud Volumes ONTAP gives end users some very serious advantages when it comes to data protection and storage management.
Trident provisions new persistent container storage that benefit from all of NetApp’s data management capabilities. Using stateful sets simplifies the deployment of all stateful applications by automating much of the required provisioning and administrative activity. If a pod in a stateful set goes down, Kubernetes will automatically bind a new instantiation of the pod to the dynamically provisioned persistent volume it was previously using.
Cloud Volumes ONTAP is deployable to either AWS storage or Azure storage and delivers:
- High availability: Cloud Volumes ONTAP HA provides storage failover capability that works across Availability Zones, and guarantees high availability, RPO=0, and RTO < 60 seconds.
- Data protection: NetApp Snapshots are used to create instant, space efficient, backups of a storage volume of any size that can be instantly restored back to the source volume at any time in the future. SnapMirror uses snapshots as the basis for incrementally replicating a volume to another instance of Cloud Volumes ONTAP, which may reside in another region.
- Storage efficiencies: Volumes created using Cloud Volumes ONTAP are able to use built-in technologies such as thin provisioning, data deduplication, and data compression making it possible to reduce cloud storage footprint and operational costs.
- Data Tiering: Kubernetes storage classes can be mapped to different pools of storage in Cloud Volumes ONTAP, which are created from any of the available AWS or Azure disk types. Cloud Volumes ONTAP can then transparently tier cold data to Amazon S3 or Azure Blob to save on storage costs.
- Storage cloning: Using NetApp FlexClone® technology, existing storage volumes can be instantly and space-efficiently cloned and made available for read/write use. Trident integrates with cloning to allow persistent volume claims to use a clone, rather than always provisioning new storage.
Stateful applications in Kubernetes require persistent storage with an independent lifecycle from their pods. This can be achieved using persistent volumes directly, however, stateful sets and dynamic provisioning provide a solution that is easier to manage and scale up. Dynamically provisioning storage for stateful sets using Trident and Cloud Volumes ONTAP provides the data protection, high availability, and flexible storage management that is a must for reliably storing enterprise information.