Kubernetes Storage

Kubernetes Persistent Volumes, Claims, Storage Classes, and More

Kubernetes Persistent Storage offers Kubernetes applications a convenient way to request, and consume, storage resources. A Volume is a basic building block of the Kubernetes storage architecture. Kubernetes Persistent Volumes are a type of Volume that lives within the Kubernetes cluster, and can outlive other Kubernetes pods to retain data for long periods of time.


Other central Kubernetes storage concepts include Persistent Volume Claims, which are requests by Kubertnetes nodes for storage resources, and Storage Classes, which define types of storage, allowing Kubernetes resources to access them without knowing their underlying implementation.


In this post, we’ll review these concepts, explain Kubernetes storage integrations and features, and show how NetApp Cloud Volumes ONTAP can help provision highly available, high performance storage for Kubernetes applications.

In this article you will learn:

What is Kubernetes Persistent Storage?

Containers are immutable, meaning that when a container shuts down, all data created during its lifetime is lost. This is suitable for some applications, but in many cases, applications need to preserve state or share information with other applications. A common example is applications that rely on databases (see our post on MySQL Kubernetes). For these and other use cases, there is a need for containers to have a place to store information persistently—so it can survive the shutdown of one or more containers. 

Kubernetes provides a convenient persistent storage mechanism for containers. It is based on the concept of a Persistent Volume (PV). Kubernetes Volumes are constructs that allow you to mount a storage unit, such as a file system folder or a cloud storage bucket, to a Kubernetes node and also share information between nodes. Regular Volumes are deleted when the Pod hosting them shuts down. But a Persistent Volume is hosted in its own Pod and can remain alive for as long as necessary for ongoing operations.

What are Kubernetes Persistent Volumes?

Kubernetes persistent volumes (PVs) are a unit of storage provided by an administrator as part of a Kubernetes cluster. Just as a node is a compute resource used by the cluster, a PV is a storage resource.

Persistent volumes are independent of the lifecycle of the pod that uses it, meaning that even if the pod shuts down, the data in the volume is not erased. They are defined by an API object, which captures the implementation details of storage such as NFS file shares, or specific cloud storage systems.

Kubernetes persistent volumes are administrator-provided volumes. They have predefined properties including file system, size, and identifiers like volume ID and name.

In order for a Pod to start using these volumes, it must request a volume by issuing a persistent volume claim (PVC). PVCs describe the storage capacity and characteristics a pod requires, and the cluster attempts to match the request and provision the desired persistent volume. 

There are two related concepts you should understand as you start working with Kubernetes persistent volumes:

PersistentVolumeClaim (PVC)

This is a request sent by a Kubernetes node for storage. The claim can include specific storage parameters required by the application—for example an amount of storage, or a specific type of access (read/write, read-only, etc.). 

Kubernetes looks for a PV that meets the criteria defined in the user’s PVC, and if there is one, it matches claim to PV. This is called binding. You can also configure the cluster to dynamically provision a PV for a claim.

StorageClass

The StorageClass object allows cluster administrators to define PVs with different properties, like performance, size or access parameters. It lets you expose persistent storage to users while abstracting the details of storage implementation. There are many predefined StorageClasses in Kubernetes (see the following section), or you can create your own.

Administrators can define several StorageClasses that give users multiple options for performance. For example, one can be on a fast SSD drive but with limited capacity, and one on a slower storage service which provides high capacity.

Persistent Volume Life Cycle

A PV is a cluster resource and a PVC is a request for a PV resource. The interaction between PVs and PVCs follows a distinct lifecycle, starting with provisioning and including binding, using, and reclaiming.

Provisioning

Here are the two main types of provisioning:

  • Static - PVs created by a cluster administrator. These PVs are located in the Kubernetes API and contain information about storage resources available to cluster users.
  • Dynamic - to meet PVC demands, clusters can attempt to dynamically provision a volume. This typically occurs when available static PVs do not match the PVC. In this case, the cluster can optionally be configured to use the default StorageClasses.

Binding

Here is a quick rundown of how the binding process works:

  • A user creates a PVC request, specifying a certain amount of storage and the required access modes.
  • A control loop, located in the master, looks for new PVCs, and then:
    • Static - the control loop attempts to find a matching PV and then binds it to the PVC.
    • Dynamic - if a PV was already dynamically provisioned for the PVC, the control loop binds them together.
  • After a PVC and a PV are bound, they remain exclusive.

A PVC to PV binding is a one-to-one mapping. The process uses a ClaimRef, which creates a bi-directional binding between the PV and the PVC.

Using

Pods use claims as volumes. Here is how the using process works:

  • The cluster inspects the claim to find the bound volume and then mounts that volume.
  • When using volumes that support multiple access modes, users can specify the desired mode.
  • Once a user is provisioned a volume, the bound PV belongs to the user.

It is also possible to schedule Pods. Users can schedule Pods and access a claimed PV by including a PersistentVolumeClaim section in the volumes block of the Pod.

Reclaiming

A reclaim policy for PVs tells the cluster what to do with the volume after the claim is released, and volumes can be Retained, Recycled, or Deleted. Once users do not need their volume anymore, they can delete the PVC objects from the API that allows reclamation of the resource.

Types of Persistent Volumes

Kubernetes comes with numerous plugins that let you make different types of storage resources available to nodes in the Kubernetes cluster. These are implemented using the StorageClass object. Here are some of the main plugins currently supported:

Cloud Storage and Virtualization

Proprietary Storage Platforms

Physical Drives / Storage Protocols

GCEPersistentDisk

Flocker

NFS

AWSElasticBlockStore

RBD (Ceph Block Device)

iSCSI

AzureFile

Cinder (OpenStack block storage)

FC (Fibre Channel)

AzureDisk

Glusterfs

 

VsphereVolume

Flexvolume

 

 

Quobyte Volumes

 

 

Portworx Volumes

 

 

ScaleIO Volumes

 

 

StorageOS

 

For more details on these plugins, see the StorageClass documentation

Read our blog post on the Kubernetes NFS integration. 

Persistent Volumes Features

Kubernetes Persistent Volumes offer powerful capabilities. The most important are detailed below.

Capacity

The capacity attribute lets you set the maximum storage capacity of the PV. Storage is specified in bytes, to ensure quantities are standard across all storage services and devices.

Volume Mode

By default, Kubernetes creates a file system on the PV, but if desired, you can use a raw block device directly without an additional layer.

Access Modes

 

A PV can have the following access modes:

●      ReadWriteOnce—enables read and write and can be mounted by only one node

●      ReadOnlyMany—enables read only and can be mounted by multiple nodes (but not at the same time)

●      ReadWriteMany—both read and write, can be mounted by several nodes (not at the same time)

 

Note: Different storage plugins may only support some of these access modes.

Reclaim Policy

The reclaim policy specifies what happens when the node no longer needs the persistent storage. It can be set to Retain, meaning the PV is kept alive until it is explicitly deleted; Recycle, meaning the data is scrubbed but can be restored later; and Delete, meaning it is irreversibly deleted.

 

Note: Different storage plugins may only support some of these reclamation policies.

Phase

A PV goes through the following lifecycle phases, which are visible to other entities in the cluster:

●      Available—free for use, binding has not occured yet

●      Bound—the PV was matched to a PersistentVolumeClaim and binding has occurred

●      Released—the user deleted their PVC, but the PV is not yet reclaimed by the cluster

●      Failed—the PV could not be reclaimed by the cluster automatically

Kubernetes Persistent Storage with Cloud Volumes ONTAP

NetApp Cloud Volumes ONTAP, the leading enterprise-grade storage management solution, delivers secure, proven storage management services on AWS, Azure and Google Cloud. Cloud Volumes ONTAP supports up to a capacity of 368TB, and supports various use cases such as file services, databases, DevOps or any other enterprise workload.


In particular, Cloud Volumes ONTAP provides Kubernetes integration for persistent storage requirements of containerized workloads.

New call-to-action

Yifat Perry, Product Marketing Lead

Product Marketing Lead

-