Azure NetApp Files
Cloud Volumes Service for AWS
Cloud Volumes ONTAP
Cloud Sync
Cloud Compliance
Cloud Tiering
SaaS Backup
Cloud Backup Service
NetApp Kubernetes
Cloud Manager
Fabric Orchestrator
Cloud Insights.svg){kind=small}
Data Migration
Cloud Native Applications
Data Protection.svg){kind=small}
File Services
Databases.svg){kind=small}
HPC & Analytics
Block Storage
SAP
Cloud First Partners
Technology Partners
Become a Partner
Jan 22, 2019, 9:51:39 AM
Kubernetes provides excellent support for stateful applications, such as MySQL, through the use of persistent volumes, which are used to create permanent data storage for pods and containers. This allows you to deploy and manage all of the constituent services that make up your enterprise architecture using a single, scalable platform.
With NetApp Trident, the underlying storage for persistent volumes can be allocated using Cloud Volumes ONTAP, which brings enterprise-grade data management to containerized applications in your Kubernetes cluster.
In this article, we discuss the benefits of deploying database storage with Cloud Volumes ONTAP, and demonstrate the way in which MySQL Kubernetes deployments can be set up using NetApp Trident.
One of the most essential requirements for deploying stateful applications is stable storage. In Kubernetes terms, this storage must survive even after a pod is rescheduled to a different node. Persistent volumes provide a solution for this by allowing storage volumes to be provisioned independently of any pod. Persistent volume claims are then used to relate persistent volumes to specific pods.
By using Kubernetes PersistentVolume reclaim policy of retain, each persistent volume can be made to keep its data intact, even when the active persistent volume claim that references it is released.
Kubernetes also provides a sophisticated mechanism for scaling stateful applications called stateful sets. This works best for horizontally scalable systems, such as NoSQL databases, where a new node with persistent storage can be deployed from a template. Using a Kubernetes stateful set ensures that the network identity of each pod will remain stable and that the persistent volumes it is bound to are never deleted, even after downscaling.
Data Protection: Stateful applications have strong data protection requirements, which includes backups, high availability, and DR (disaster recovery) capabilities. The internal storage provisioners that come with Kubernetes enable pods to interface with a variety of storage services, such as Amazon EBS and Azure Disk, however, ensuring that the data being stored is protected against all types of failures is largely responsibility of the end-user administrators setting up the platform.
DevOps Datasets: In order to support faster TTM (Time to Market) for enterprise applications and services, DevOps engineers need to create software test environments and CI/CD pipelines that may also require up-to-date copies of production databases, which, due to the size of the datasets involved, may be difficult and time consuming to re-create or restore from backup.
Cloud Volumes ONTAP is NetApp’s cloud-based data management platform for AWS and Azure. Building on native cloud compute and AWS and Azure storage resources, Cloud Volumes ONTAP offers a comprehensive set of features for working with data storage, from storage space efficiencies that dramatically reduce cloud storage costs to enterprise-grade data protection. Containerized applications deployed to Kubernetes can benefit from these features by using the NetApp Trident storage provisioner, which enables persistent volumes to be created directly in Cloud Volumes ONTAP.
For database services, Cloud Volumes ONTAP delivers critical data protection features. For example, ONTAP Snapshots™ provide instant backup and instant restore for databases of any size. These snapshots are also very storage efficient, requiring only enough space to keep them consistent with the point-in-time to which they relate. Creating snapshots with NetApp SnapCenter® ensures that database I/O is properly quiesced beforehand, which is necessary for reliable backups.
Cloud Volumes ONTAP HA provides high availability across Availability Zones for Kubernetes persistent volumes, which improves upon the redundancy within an Availability Zone usually provided by iSCSI cloud storage solutions. In the event of a failure, the storage can be seamlessly failed over to provide uninterrupted access for clients and applications actively using the storage.
NetApp SnapMirror® provides efficient, block-level data replication between ONTAP storage services, whether on-premises or in the cloud with AWS or with Azure. This allows for persistent volumes to be mirrored either for DR purposes or archived for long-term backups, to a secondary location that may exist in a different region. After making an initial baseline copy of the source data to the destination, each subsequent synchronization only transfers the changes made to the source data since the last time it ran.
There are many additional advantages to using Cloud Volumes ONTAP for database storage. One of these is storage tiering, which is the ability to automatically and transparently move cold data to low cost object storage, such as Amazon S3 and Azure Blob. On access, the data is moved back to the performance tier for faster access. This feature can be used to make cloud storage more cost effective for large databases that contain a lot of infrequently used, historical data.
Another extremely useful feature is NetApp FlexClone®, which gives DevOps and database administrators the power to instantly clone existing persistent volumes without needing to make an additional copy of the data. This feature has been integrated into NetApp Trident, which allows clones to be created through persistent volume claims. Cloning makes it very easy to set up temporary, writable copies of a database for testing.
MySQL is a popular flavor of SQL database. The following provides an example of deploying a single Kubernetes MySQL instance using NetApp Trident.
First, we will create a persistent volume claim for the storage we need. As Trident uses dynamic provisioning, we will specify a storage class, which must have been setup prior to executing this manifest. Each storage class defines the provisioner to be used, along with any other provisioner-specific settings that will determine how the storage is provisioned.
When this claim is used by a pod, Trident will automatically create a 100 GiB high-performance storage volume in Cloud Volumes ONTAP to fulfil the request. This storage class may have been implemented to make use of Amazon EBS Provisioned IOPS disks, for example. A reclaim policy of “retain” has also been specified, which will prevent the persistent volume from being deleted if the pod releases the claim.
We can now bind a pod to this storage by referencing it in our MySQL deployment manifest. As shown below, we simply need to reference our previously defined persistent volume claim and specify where the bound persistent volume should be mounted within our MySQL Kubernetes container.
We can additionally set up test instances of our Kubernetes MySQL database using Trident’s integration with NetApp FlexClone. The following persistent volume claim manifest demonstrates how this can be done.
We simply annotate our claim with the name of the persistent volume claim to clone from, and Trident takes care of the rest by automatically finding the source storage volume in Cloud Volumes ONTAP and performing a NetApp FlexClone operation. The default reclaim policy of delete will ensure that the clone volume is automatically deleted when it is no longer required.
Kubernetes provides a complete solution for running both stateless and stateful applications, with the extensible flexibility to integrate with various cloud storage solutions through its persistent volumes and provisioner paradigms. NetApp Trident makes it possible to integrate Kubernetes with Cloud Volumes ONTAP, giving access to a whole host of data protection and data management benefits that are vital for running database platforms, including MySQL Kubernetes deployments.
Find out more about Cloud Volumes ONTAP with Kubernetes to revolutionize the way you use and manage container storage, or start a free 30-day trial today in with AWS or Azure.
With NetApp Trident, the underlying storage for persistent volumes can be allocated using Cloud Volumes ONTAP, which brings enterprise-grade data management to containerized applications in your Kubernetes cluster.
In this article, we discuss the benefits of deploying database storage with Cloud Volumes ONTAP, and demonstrate the way in which MySQL Kubernetes deployments can be set up using NetApp Trident.
How Does Kubernetes Storage Work?
One of the most essential requirements for deploying stateful applications is stable storage. In Kubernetes terms, this storage must survive even after a pod is rescheduled to a different node. Persistent volumes provide a solution for this by allowing storage volumes to be provisioned independently of any pod. Persistent volume claims are then used to relate persistent volumes to specific pods.
By using Kubernetes PersistentVolume reclaim policy of retain, each persistent volume can be made to keep its data intact, even when the active persistent volume claim that references it is released.
Kubernetes also provides a sophisticated mechanism for scaling stateful applications called stateful sets. This works best for horizontally scalable systems, such as NoSQL databases, where a new node with persistent storage can be deployed from a template. Using a Kubernetes stateful set ensures that the network identity of each pod will remain stable and that the persistent volumes it is bound to are never deleted, even after downscaling.
Kubernetes Storage Challenges
Data Protection: Stateful applications have strong data protection requirements, which includes backups, high availability, and DR (disaster recovery) capabilities. The internal storage provisioners that come with Kubernetes enable pods to interface with a variety of storage services, such as Amazon EBS and Azure Disk, however, ensuring that the data being stored is protected against all types of failures is largely responsibility of the end-user administrators setting up the platform.
DevOps Datasets: In order to support faster TTM (Time to Market) for enterprise applications and services, DevOps engineers need to create software test environments and CI/CD pipelines that may also require up-to-date copies of production databases, which, due to the size of the datasets involved, may be difficult and time consuming to re-create or restore from backup.
Setting up Kubernetes MySQL in Kubernetes with Cloud Volumes ONTAP
Cloud Volumes ONTAP is NetApp’s cloud-based data management platform for AWS and Azure. Building on native cloud compute and AWS and Azure storage resources, Cloud Volumes ONTAP offers a comprehensive set of features for working with data storage, from storage space efficiencies that dramatically reduce cloud storage costs to enterprise-grade data protection. Containerized applications deployed to Kubernetes can benefit from these features by using the NetApp Trident storage provisioner, which enables persistent volumes to be created directly in Cloud Volumes ONTAP.
For database services, Cloud Volumes ONTAP delivers critical data protection features. For example, ONTAP Snapshots™ provide instant backup and instant restore for databases of any size. These snapshots are also very storage efficient, requiring only enough space to keep them consistent with the point-in-time to which they relate. Creating snapshots with NetApp SnapCenter® ensures that database I/O is properly quiesced beforehand, which is necessary for reliable backups.Cloud Volumes ONTAP HA provides high availability across Availability Zones for Kubernetes persistent volumes, which improves upon the redundancy within an Availability Zone usually provided by iSCSI cloud storage solutions. In the event of a failure, the storage can be seamlessly failed over to provide uninterrupted access for clients and applications actively using the storage.
NetApp SnapMirror® provides efficient, block-level data replication between ONTAP storage services, whether on-premises or in the cloud with AWS or with Azure. This allows for persistent volumes to be mirrored either for DR purposes or archived for long-term backups, to a secondary location that may exist in a different region. After making an initial baseline copy of the source data to the destination, each subsequent synchronization only transfers the changes made to the source data since the last time it ran.
There are many additional advantages to using Cloud Volumes ONTAP for database storage. One of these is storage tiering, which is the ability to automatically and transparently move cold data to low cost object storage, such as Amazon S3 and Azure Blob. On access, the data is moved back to the performance tier for faster access. This feature can be used to make cloud storage more cost effective for large databases that contain a lot of infrequently used, historical data.
Another extremely useful feature is NetApp FlexClone®, which gives DevOps and database administrators the power to instantly clone existing persistent volumes without needing to make an additional copy of the data. This feature has been integrated into NetApp Trident, which allows clones to be created through persistent volume claims. Cloning makes it very easy to set up temporary, writable copies of a database for testing.
How to Set Up MySQL Databases
MySQL is a popular flavor of SQL database. The following provides an example of deploying a single Kubernetes MySQL instance using NetApp Trident.
First, we will create a persistent volume claim for the storage we need. As Trident uses dynamic provisioning, we will specify a storage class, which must have been setup prior to executing this manifest. Each storage class defines the provisioner to be used, along with any other provisioner-specific settings that will determine how the storage is provisioned.
When this claim is used by a pod, Trident will automatically create a 100 GiB high-performance storage volume in Cloud Volumes ONTAP to fulfil the request. This storage class may have been implemented to make use of Amazon EBS Provisioned IOPS disks, for example. A reclaim policy of “retain” has also been specified, which will prevent the persistent volume from being deleted if the pod releases the claim.
We can now bind a pod to this storage by referencing it in our MySQL deployment manifest. As shown below, we simply need to reference our previously defined persistent volume claim and specify where the bound persistent volume should be mounted within our MySQL Kubernetes container.
We can additionally set up test instances of our Kubernetes MySQL database using Trident’s integration with NetApp FlexClone. The following persistent volume claim manifest demonstrates how this can be done.
We simply annotate our claim with the name of the persistent volume claim to clone from, and Trident takes care of the rest by automatically finding the source storage volume in Cloud Volumes ONTAP and performing a NetApp FlexClone operation. The default reclaim policy of delete will ensure that the clone volume is automatically deleted when it is no longer required.
Conclusion
Kubernetes provides a complete solution for running both stateless and stateful applications, with the extensible flexibility to integrate with various cloud storage solutions through its persistent volumes and provisioner paradigms. NetApp Trident makes it possible to integrate Kubernetes with Cloud Volumes ONTAP, giving access to a whole host of data protection and data management benefits that are vital for running database platforms, including MySQL Kubernetes deployments.
Find out more about Cloud Volumes ONTAP with Kubernetes to revolutionize the way you use and manage container storage, or start a free 30-day trial today in with AWS or Azure.