More about Kubernetes Storage
- Azure Kubernetes Service: Configuring Persistent Volumes in AKS
- Kubernetes NFS: Two Quick Tutorials
- AWS Kubernetes Cluster: Quick Setup with EC2 and EKS
- Kubernetes Shared Storage: The Basics and a Quick Tutorial
- AWS ECS vs Kubernetes: An Unfair Comparison?
- Kubernetes Persistent Storage: Why, Where and How
- Kubernetes for Developers: A Deep Dive
- Using Cloud Manager for Kubernetes Deployment
- Docker Volume Tutorial - Using Trident to Provision Storage
- How to Set Up MySQL Kubernetes Deployments
- Kubernetes Persistent Volumes Cloning
- Storage Efficiency for Improving Persistent Volume Storage Costs
- Protection for Persistent Data Storage in Kubernetes
- Dynamic Kubernetes Persistent Volume Provisioning
- Managing Stateful Applications in Kubernetes
- Understanding Kubernetes Persistent Volume Provisioning
- An Introduction to Kubernetes
The NFS (Network File System) protocol is widely used to share files in enterprise environments, allowing many users to access the same files at the same time. This makes sharing data a lot easier, and no less so in cloud file sharing.
NFS can be used for a wide range of use cases, such as for creating data lakes for building analytics solutions, database services, data archives, etc. If you’re using Kubernetes, NFS can be used with Kubernetes pods to manage persistent storage requirements and share data and files between containers and other pods.
In this post we’ll take a look at the advantages of using NFS in Kubernetes environments. We’ll also show you how to use NetApp’s Trident and Cloud Volumes ONTAP as an enterprise-grade Kubernetes NFS provisioner. This will include some provisioning and Kubernetes NFS storage class examples in code, plus, explaining the data management benefits that come along with using Cloud Volumes ONTAP.
Using NFS File Service from Kubernetes
Let’s first take a look at some of the reasons why you’d use shared storage in a Kubernetes cluster.
For starters, there’s an advantage to NFS over iSCSI because managing a large number of individual iSCSI storage allocations can add to administrative overhead. All those block-level persistent volume allocations can make storage management more difficult, especially when Kubernetes persistent volumes are statically allocated. In these cases, storage utilization may be lower, as a persistent volume is used only for a single pod and that pod may not make use of all of its storage. Also, sharing data between pods can be difficult when using block-level persistent volumes.
Unless containers are deployed to the same pod, sharing data between containers can also be difficult. Deploying containers in this way, i.e. to the same pod, should only be done when it makes sense for the application, such as when the containers work together directly to fulfill some function. Putting containers into the same pod when they simply need to share data can lead to scalability problems as a pod will only be scheduled to a single node.
NFS solves these problems by allowing many hosts to mount the same file system at the same time, and for all hosts to access files concurrently. With NFS, users don’t need to format the storage volume using an Operating System file system, such as ext4; the storage can simply be mounted and used straight away. This makes it much easier to attach storage to pods and reduce the administrative overhead of working with persistent storage. NFS storage volumes in Kubernetes can also be easily expanded without any client-side changes using Trident, which we’ll discuss in more detail in the next section.
NFS Persistent Volume Provisioning with Trident
NetApp Trident is a storage provisioner for Kubernetes that allows users to take advantage of NetApp storage services, both on-premises and in the cloud. Trident is a fully-supported, open-source solution that allows native Kubernetes manifests to be used to provision persistent volumes via Cloud Volumes ONTAP.
As a Kubernetes NFS provisioner, the benefits of using Trident include mounting persistent volumes as Read/Write Many, dynamically resizing NFS persistent volumes, and creating separate storage classes for different mount parameters and other requirements.
Mounting Persistent Volumes as Read/Write Many
Persistent Volumes using NFS can be mounted as Read/Write Many, which allows multiple pods to mount the same volumes. Using native Kubernetes constructs means that storage can be allocated without the need for custom procedures or processes. Using the Read/Write Many access mode allows many persistent volumes claims to access the same persistent volume.
With block-level Kubernetes persistent volumes, Read/Write Once must be used, which means that there is a 1-to-1 relationship between persistent volume claims and persistent volumes.
Here’s a persistent volume claim (PVC) for a Kubernetes NFS volume example:
Dynamically Resizing NFS Persistent Volumes
New to provisioning in Kubernetes is the support for editing persistent volumes claims to request more storage space. With this new capability, persistent volumes can be expanded without needing to re-create the pod that uses the storage. In order for that to happen, the underlying storage provisioner must support resize. The commonly used provisioners, such as those for Amazon EBS or Azure Disk storage support this—except not when it comes to NFS. With Trident, NFS persistent volumes can be dynamically resized and all the back-end changes required taken care of. This makes Trident a major value add.
By extending the support for resizing persistent volumes to NFS, Trident is a major value add.
Block-level storage, such as Amazon EBS, needs to have the file system extended when the underlying storage is expanded. Kubernetes will take care of this, but it will also require that the pod using the storage to be restarted. That will cause downtime. With NFS, no file system expansion is required, and the pod can continue to work without interruption.
You can read more about resizing NFS volumes with Trident here.
Creating Separate Storage Classes
Separate storage classes can be created for different mount parameters and other requirements, for example using separate storage classes for NFSv3 and NFSv4.
Storage classes allow users to control the type and configuration of the storage they are using when provisioning in Kubernetes. That extends to the Kubernetes dynamic provisioning NFS storage Trident provides.
Take this Kubernetes NFS storage class example. Here is the storage class manifest:
The allowVolumeExpansion attribute controls whether the resizing of persistent volumes created using this storage class is allowed or not.
The storage class can be used to control the storage pool, or NetApp aggregate used to provision persistent volumes, e.g. a storage class named gold may be used to configure high performance back-end storage, where as one called bronze may be configured for capacity storage.
The Added Benefits of Using Cloud Volumes ONTAP
Cloud Volumes ONTAP provides a robust and scalable solution for deploying and managing storage in the AWS or Azure cloud. NFS storage volumes provisioned with Cloud Volumes ONTAP benefit from the whole feature set available, such as cost-saving storage efficiencies, NetApp Snapshots™ for data protection, cloning to create instant, zero-capacity test copies with FlexClone®, and SnapMirror® for data replication and redundancy across regions. In addition to NFS, Cloud Volumes ONTAP can also serve the NAS protocols SMB/CIFS, as well as iSCSI.
The data protection benefits of NetApp Snapshots and snapshot-based syncs are a major benefit that you won’t get using Amazon EFS, and that is more efficient than the native solutions on Azure Files or the new Amazon FSx, and which can also benefit from Cloud Volumes ONTAP’s storage efficiencies.
Plus, the Cloud Volumes ONTAP HA option offers multi-node data protection and high availability in both AWS and in Azure. That means no data loss (RPO=0) and minimal downtime (RTO < 60 seconds) if you experience a failure or need to shut down a node for maintenance.
Using NFS for shared storage can be an attractive option because it can be easier to manage and more flexible than other storage formats, such as iSCSI. NetApp Trident and Cloud Volumes ONTAP make is easier to deploy and manage NFS for Kubernetes clusters in the cloud.
Using Trident as your Kubernetes NFS provisioner gives you some additional benefits that you won’t get with other provisioners, such as the ability to dynamically resize NFS persistent volumes. Combining that with Cloud Volumes ONTAP’s powerful data storage solutions, such as storage efficiency, volume cloning, and data replication, makes for an enterprise solution for Kubernetes and NFS.
For more on file shares, take a look at the cloud file sharing hub on NetApp Cloud Central. You’ll find blogs about NFS options in the cloud with Amazon EFS and Google Cloud Filestore, consolidating unstructured data with Cloud Volumes ONTAP and Talon FAST™, configuring volume access, customer case studies, and more.