hero-testdrive-graphic.png

Blog

Storage Tiering with NetApp ONTAP Cloud and Amazon S3

Posted by Gali Kovacs on Sep 26, 2017 3:43:31 AM

Data Storage Tiering

ONTAP Cloud deployments in AWS make use of Amazon EBS volumes as the underlying cloud storage layer for delivery of its enterprise-class data management services. These volumes come in a variety of types which you can choose between based on your requirements for performance and capacity. 

Now, with the advent of storage tiering for ONTAP Cloud, NetApp extends this flexibility by enabling you to use Amazon S3 as a capacity storage tier for your “cold” data.

Storage tiering for ONTAP Cloud works in by using Amazon EBS as your performance tier for your “hot” data in conjunction with Amazon S3 as the capacity tier for your “cold” data.

ONTAP Cloud automatically moves data between these tiers as required, which dramatically reduces costs for cold, infrequently-accessed data, while still providing rapid access to hot data. Having the ability to store the large bulk of your data in lower-cost Amazon S3 storage, until it’s actually needed, is ideal for setting up cloud-based DR environments.

In this article we’ll look at how ONTAP Cloud storage tiering works, how to set it up, the DR use case for the technology, and the costs benefits when compared to Amazon EBS.

Capacity Tiering with Amazon S3

Capacity Tiering Amazon S3

Starting with ONTAP 9.2, storage volumes can be created on flash disk aggregates that are backed by tiered object storage architecture with the addition of the FabricPool feature. For ONTAP Cloud, these aggregates can use either General Purpose SSD (gp2) or Throughput Optimized HDD (st1) EBS volumes as the performance tier, with Amazon S3 as the capacity tier.

When a volume is created on such as aggregate, ONTAP Cloud will automatically move data that it considers cold over to Amazon S3 and free up the associated Amazon EBS storage. If the data stored on the Capacity Tier (Amazon S3) is accessed, it is transparently pulled back to the Performance Tier (Amazon EBS). Each volume therefore has part of its storage allocation in Amazon EBS and the other part in Amazon S3.

Tiering Demo

Setting up ONTAP Cloud storage tiering is very simple. The only requirement is to create a VPC endpoint to the Amazon S3 service. This ensures secure, encrypted communication to and from your Amazon S3 buckets over the AWS network. ONTAP Cloud automatically takes care of the rest, including the allocation of an Amazon S3 bucket for your data. There’s no further manual configuration or tweaking required—it just works.

Storage Tiering Amazon S3

Cloud Disaster Recovery Use Case

When setting up a DR environment, storage is required for a lot of data that would normally be accessed only in an emergency, either when there has been some kind of disaster or catastrophic system failure.

In the ideal case, we’d be able to store this data on lower-cost storage while it is not being actively used, and move it back to high performance storage when it actually needs to be accessed. This would reduce the upfront cost of setting up our cloud-based DR environment by deferring the cost of allocating high performance Amazon EBS storage until it’s actually needed. Using the new ONTAP Cloud storage tiering feature, we can do exactly that.

SnapMirror is NetApp’s solution for efficient and incremental storage replication, which can be used to synchronize data from on-premises NetApp appliances to ONTAP Cloud, such as would be required for a cloud-based DR environment.

When a SnapMirror destination volume is stored on a tiered aggregate, ONTAP Cloud will automatically push the majority of the data to Amazon S3, retaining only metadata information about the volume in Amazon EBS. This remains the case for new data that arrives incrementally.

If the SnapMirror relationship is broken and the destination volume are accessed, ONTAP Cloud will transparently pull only the data that are required from Amazon S3 back into Amazon EBS. If the SnapMirror relationship is subsequently re-established, the data that was pulled onto the Amazon EBS performance tier will be transparently moved back to the Amazon S3 capacity tier, freeing up the space used in the performance tier.

This data movement to and from the Amazon EBS and S3 tiers is managed automatically by ONTAP Cloud without any user intervention required.

Cost Comparison

One of the main drivers for using ONTAP Cloud storage tiering is the reduction in cloud storage costs for infrequently-accessed data.

Let’s now consider a simple example in which 10TB of such storage is required. The monthly storage cost comparison is given in the table below.

 

Amazon EBS (gp2)

Amazon EBS (st1)

Amazon S3 Standard

Capacity

10TB

10TB

10TB

Pricing

$0.10 per GB-month

$0.045 per GB-month

$0.023 per GB

Cost / Month

$1024

$460.80

$235.52

 

The AWS region used for this pricing was US East (N. Virginia), with the assumption that the storage would be allocated for an entire month. In this AWS region, the price for S3 Standard is lower when more storage is allocated, falling to $0.022 per GB for storage over 50TB and $0.021 for allocated storage over 500TB.

The costs above relate purely to data storage, as there are also access request costs related to Amazon S3. However, as the Amazon S3 tier will be storing cold data, less access to the data would normally be expected.

As we can see, there is a huge price differential between using Amazon EBS and Amazon S3. Storage costs for Amazon S3 Standard are a quarter of the cost of General Purpose SSD (gp2). The cost savings for Throughput Optimized HDD (st1) are also significant, with Amazon S3 costs being roughly half the price. This makes for a very tangible reduction in cloud storage costs for an ONTAP Cloud environment.

Summary

NetApp has further enhanced data management in the cloud by offering the ability to seamlessly tier cold data to and from less expensive Amazon S3 capacity storage. In this article, we’ve shown how easy this is to set up and use and the huge savings in cloud storage costs that result, especially ONTAP Cloud DR environments.

As well as data replicated using SnapMirror, ONTAP Cloud storage tiering is also able to move infrequently-accessed data from ONTAP Snapshots over to Amazon S3. This, again, happens automatically when the snapshot data is deemed to be cold. In the future, support will also extend to the tiering of cold data in active filesystems.

To try out ONTAP Cloud, start a 30-day free trial and claim $100 of free AWS credits to get you started.

AWS Storage Options

Related Articles:

Topics: Cloud Storage