Apr 18, 2018 12:27:00 AM
Not all data needs to be stored in the highest performing storage format. The cool and archive Azure storage tiers available in Azure Storage are meant to be used to retain data that is infrequently accessed. In general, that means any data, such as research raw data, network telemetry dumps etc., that is not regularly needed cannot be removed from the system as it might be required for future reference.
While the most critical enterprise workloads run on Azure premium storage, investing in costly on-premises storage systems or high performance cloud storage for that kind of data is not economical in the long run. That’s where Azure cold storage and archive storage options come in, which can help you with lower Azure storage costs.
In this post we will focus on the different features of Azure Storage’s cool and archive tiers and show you why you should incorporate them into your IT infra-architecture as a solution for low-cost data storage. We’ll also look at some NetApp cloud solutions that can help you further save on Azure storage costs.
Azure Cool Storage Tier: Azure Cold Storage Costs LessBefore we discuss the Azure cold storage option, a bit of background. Azure cloud storage can now be created under three categories: General-purpose v1 (GPv1), General-purpose v2 (GPv2) and Blob storage.
GPv1 supports all Azure storage options such as Blobs, Files, Queues and Tables. Data tiering into hot and cold tiers was previously supported only in Azure Blob storage. General-purpose v2 storage was announced in December 2017 and supports all storage options along with tiering of Blob storage into hot, cold, and archive tiers.
Azure’s cool storage tier, also known as Azure cool Blob storage, is for infrequently-accessed data that needs to be stored for a minimum of 30 days. Typical use cases include backing up data before tiering to archival systems, legal data, media files, system audit information, datasets used for big data analysis and more.
The storage cost for this Azure cold storage tier is lower than that of hot storage tier. Since it is expected that the data stored in this tier will be accessed less frequently, the data access charges are high when compared to hot tier. There are no additional changes required in your applications as these tiers can be accessed using APIs in the same manner that you access Azure storage.
When it comes to Azure storage performance, both cool tier and hot tier have a rate of 60 Mib per second or up to 500 requests per second. The availability for cool tier is 99%, which is slightly less than the 99.9% available for hot tier. For Read-only Geo-redundant Storage (RA-GRS), the availability for cool tier is 99.9%, whereas that of hot tier is 99.99%. These differences are mainly due to the nature of the data being stored in Azure cold storage tier, where the data will be less affected by the availability targets.
Azure cool tier is equivalent to the Amazon S3 Infrequent Access (S3-IA) storage in AWS that provides a low cost high performance storage for infrequently access data.
Azure Archive Storage TierThe Azure archive storage tier, as the name indicates, is a low-cost storage for archival data that will be rarely accessed. This tier is meant for data that will remain in archival storage for a minimum of 180 days. Long-term backup data, medical history, business transactions logs for audit and compliance, call center recordings are just some use cases that could benefit from the lengthy retention period on Azure archive storage.
Even though Azure archive storage offers the lowest cost in terms of data storage, its data retrieval charges are higher than that of hot and cool tiers. In fact, the data in the archive tier remains offline until the tier of the data is changed using a process called hydration. The process of hydrating data in the archive storage tier and moving it to either hot or cool tier could take up to 15 hours and, hence, it is only intended for data that can afford that kind of access delay.
Azure also does not provide any availability SLAs for archive tier. The resiliency of the storage format is assured by replication at storage level. It supports Locally-redundant storage (LRS), Geo-Redundant storage (GRS) and RA-GRS. If you opt for GPv2 storage, Zone-redundant storage (ZRS) is also available in preview.
The counterpart of Azure Storage Archive tier in AWS is Amazon S3 Glacier, which is a long-term, secure and durable object storage.
Differentiating Cool and Archive TiersCool tier is for the kind of data that you want to store on a short-term basis but may not access on a day-to-day basis, such as for banking transactions history, e-commerce transactions history, etc. You need the data to be online, should there be a requirement to retrieve it. The speed at which the data can be accessed if a requirement arises is almost same as that of the hot storage tier.
One common use case for the cool storage tier is short-term data backup which could, at later point, be moved to a hot tier for restore or eventually moved to an archive tier for long-term storage. It can also be used for storing analytics data, large media files, telemetry, etc. The nature of the data is usually such that they would need a high capacity but low-cost storage option. Cool tier fits the bill in such cases.
Archive tier targets data that can be stored offline on a long-term basis. There will not be an immediate or urgent requirement to access the data. If a scenario arises where the data should be accessed, a wait time of up to 15 hours should be acceptable. A possible example is retrieval of medical records from an archive for research purposes. Unlike cool tier, the archive tier is not enabled at the Azure Storage level. However, it supports blob-level tiering where the archival tier can be set at object level.
General availability of Archive tier was announced in December 2017 and the service is not yet available in all Azure regions. One common use case for archive tier is long-term data retention for compliance. Low-cost storage for data sets such as patient records in hospitals, offsite Azure blob backup and raw media files are other possible scenarios where archive tier can be leveraged.
In situations where the data access patterns are unclear, it is recommended to start with hot storage tier, monitor for a while, and then move the data to cool storage or archive storage tier based on access frequency.
The Cost-Benefit AnalysisArchive tier offers the lowest Azure storage costs available on the Azure cloud at just $0.002 per GB up to first 50 TBs. However, reading data from an archive tier can be a costly activity which charges $5 for every 10,000 read operations. An early deletion charge will be applicable effective March 1 2018 for any data that was deleted before the minimum period of 180 days for archive tier.
Cool tier data storage costs $0.01 GB up to 50 TB, which is higher than that of archive tier. However, there are no high charges associated with data read operations. 10,000 operations will cost only $0.01 in the cool tier.
As it can be seen from the associated costs listed above, selecting the optimal low-cost storage depends mainly on the size of data and the usage pattern. Terabytes of data can be stored at very low cost in the Archive tier. However, if a requirement arises where the data will be read more frequently, it is always economical to rehydrate and store that data in the cool tier or hot tier.
Integration with Existing Storage SystemsAzure cool and archive tier integrates into several industry leading storage service vendors to provide hybrid cloud storage offerings. The frequently-accessed data will be stored in on-premises storage or to the hot tier and the lesser frequently accessed data is tiered to cool or archive tier in Azure. It could initially be in cool tier, and at later point can be moved to archive tier based on access patterns.
The native offering from Microsoft in this space is the StorSimple solution that supports cool tier. Different flavors of the solution such as StorSimple physical array, Virtual Array and cloud array has this integration available out of the box.
The Added Value of Cloud Volumes ONTAP (formerly ONTAP Cloud)NetApp offers out-of-the-box cloud storage integration with Cloud Volumes ONTAP for Azure storage. Cloud Volumes ONTAP interoperates with Azure cloud storage to cater to multiple use cases such as enterprise workload hosting, disaster recovery with Azure Site Recovery, DevOps and as a file share service.
Depending on the workloads to be supported, customers have the option to choose from different storage types available in Azure, such as HDD or SSD-backed storage. In either case, Cloud Volumes ONTAP helps with Azure cost optimization, lowering Azure storage costs up to 70% with space-saving storage efficiencies and data tiering to Azure blob storage..
NetApp’s hybrid cloud solution helps enterprise customers reap the benefits of both on-premises as well as cloud storage features of cool and archive Azure storage. To see how using Cloud Volumes ONTAP can cut down your TCO on Azure, check out our free Azure calculator.
Many organizations are addressing the exponential data growth and backup data storage challenge by leveraging cloud storage or hybrid cloud storage solutions. Cloud storage is a cheaper option when compared to on-premises data storage costs. Further cost reductions can be achieved by leveraging tiered low-cost Azure storage options for least-accessed data.
Azure hot and cold tier fits are inexpensive Azure cloud storage variants that fit into this space, space that NetApp’s Cloud Volumes ONTAP can further help in cost reduction with its range of storage-efficiency features. It is highly recommended that organizations include them in their environment storage architecture.
For more ways to cut down on storage costs, consider a free 30-day trial of Cloud Volumes ONTAP for Azure today.