Blog

Storage Tiering to Amazon S3 or Azure Blob with Cloud Volumes ONTAP

Cost is a major concern for any company that manages its storage in the cloud or in a hybrid cloud deployment. One way to pay less for storing large amounts of data is to tier infrequently-used data to less-expensive storage formats. However, the public cloud providers only offer that service between tiers of the object storage format. How can companies get more savings by tiering between block storage and object storage?

Cloud Volumes ONTAP data tiering makes it possible to automatically and seamlessly move data between block storage and object storage on Amazon S3 or Azure Blob. It also has the ability to take advantage of the specific storage class levels within each object format service. In this post, we’ll take a closer look at the data tiering capabilities of Cloud Volumes ONTAP.

What is Data Tiering?


Data tiering makes it possible to move data between different storage types and layers. The main reason for this would be to take advantage of lower-latency storage types for data that is immediately required (what we will call “hot data”) and to move that data to higher latency, less-expensive storage types when it is not needed for immediate use (this we’ll call “cold data”). Any data can qualify as cold data. It can be the “usual” infrequently used data such as snapshots, DR environments, long term backups, etc. but also active, live data that isn’t accessed that frequently.

The cloud providers’ tiering capabilities are limited within object storage tiers only, but with Cloud Volumes ONTAP, cold data can be moved automatically, seamlessly, and quickly between highly-performant block storage tier on Amazon EBS or managed Azure Disks and object storage capacity tier in Amazon S3 or Azure Blob. Since object storage costs are significantly lower than the costs of block storage, great cost savings are expected with this capability. This data tiering function can also be leveraged to work around DBaaS storage limitations by sending the less-frequently used data to object storage tier.

How to Tier Data with Cloud Volumes ONTAP


Storage Tiering with Cloud Volumes ONTAPCloud Volumes ONTAP tiers data from performant block storage on Amazon EBS or Azure Disks and shifts it to object storage on Amazon S3 or Azure Blob, respectively. The additional tiering levels within Amazon S3 and Azure Blob storage can be taken advantage of as well when using Cloud Volumes ONTAP storage tiering. In this section, we’ll see how all of these tiering capabilities work.

If you choose to tier data with Cloud Volumes ONTAP there are three different policies for doing so: Auto, Snapshot Only, and DR. Each volume that you own can have its own unique data tiering policy, which enables you to target how and why you tier data on it.

Auto


With the Auto tiering option, the cold data will be tiered to object storage. Cold data to be tiered will be both active file system data and snapshot copies. In Auto tiering, there is a default cooling period of around 31 days, which can be changed by the user.

When random reads are detected by Cloud Volumes ONTAP, the data will be termed to be hot and brought back up to the performance tier automatically. Sequential reads, such as those done by virus scans, will be ignored and the data will remain cold and in object storage.

Snapshot Only


Snapshot data is automatically tiered to the capacity tier. The cooling period in this option is around 2 days but is configurable by the user. If this snapshot data is read, the data will be considered hot and automatically moved back to the performance tier.

DR


This setting tiers the most data out of all the options and is for disaster recovery or long-term retention data copies. If activated through a read operation, this data will gradually move back to the performance tier.

Tiered Storage on AWS


With Cloud Volumes ONTAP for AWS storage, the performance tier is located on Amazon EBS and the capacity tier on Amazon S3. The capacity tier can be changed between the different AWS storage tiers used by Amazon S3. Initially, the data is sent to Amazon S3 Standard by default. However, if you do not plan to access the cold data frequently, you can further reduce your Amazon S3 storage costs by changing a system’s tiering level to Standard-Infrequent Access (Standard-IA) or One Zone-Infrequent Access. These tiers have lower storage rates, though data retrieval costs are higher. One Zone-Infrequent Access also provides less redundancy. Note that storage tiering does not function with the Amazon S3 free tier.

To see the business case for Cloud Volumes ONTAP storage tiering on AWS, consider the following. Provisioned storage for Amazon EBS General Purpose SSD in the US-East region would cost $0.10 per GB-month. That means keeping 10 TB of a disaster recovery environment data would cost $12,000 annually. By tiering the 10 TB to Amazon S3 with One Zone-IA, the least expensive of the AWS storage tiers, where storage rates are the lowest, normal storage for the data without access would cost just $1,200 per year (excluding the 30 days on Standard storage during the cooling period).

Tiered Storage on Azure


Cloud Volumes ONTAP for Azure storage takes advantage of different Azure storage tiers. Azure managed disks can be used as the performance tier and the capacity tier is on Azure Blob storage. Cloud Volumes ONTAP supports two of the Azure Blob storage tiers: the Hot access tier and the Cool access tier.

**Note that these branded terms for Azure storage tiers are not the same as what Cloud Volumes ONTAP considers to be a hot and cold tier; both the Hot access and Cool access tiers on Azure Blob are considered parts of the cold tier used by Cloud Volumes ONTAP.

More on Azure object storage tiers: The Hot access tier provides low access costs for data that can be expected to need access at some point for read/write operations. Cool access tier offers even lower rates for storage than the Hot access tier, but transaction and access costs are higher than on Hot access tier.

Cloud Volumes ONTAP will use the Hot access tier by default as the capacity tier, however users can change their tiering level to the Cool access tier to further reduce costs. Doing so will first store the data in the Hot access tier, and if the data remains un-accessed there for 30 days (or another period set by the user), it will automatically shift to the Cool access tier.

To see the business case for data tiering with Azure, consider the following. In the East US region, a Premium SSD of 32 GiB costs $5.28 per month. To store 10 TB of data that would require 292 disks, costing $1,541.61 per month and around $18,500 annually. Tiering the 10 TB of data to Azure Blob’s Cool access tier, where data costs $0.0152 per GB per month, and leaving it unaccessed would cost just $1,867 per year.

Conclusion


Data is what organizations thrive on, so it’s important that the data that is most important is available for rapid, performant use. However, storing data in disk types that enable that type of performance comes at a cost. These costs can be mitigated by tiering infrequently-used data to object storage, where the price for storage is a fraction of the cost of block storage, with a latency that fits data not expected to be used often.

With Cloud Volumes ONTAP, you can create a tiered storage architecture for seamless and automated storage tiering between AWS storage tiers or Azure storage tiers. This makes sure that you not only save costs on storage but also eliminate timely and possibly costly errors in moving data between tiers.

Try Cloud Volumes ONTAP free for 30 days on AWS or 30-day Azure free trial.

Faster Genome-Sequence Analysis
-