Blog

All About AWS EBS (Plus Five AWS EBS Functions You Aren’t Using)

To successfully meet the challenge of storing data in the cloud using AWS, IT professionals need to familiarize themselves with the management of Amazon Elastic Block Store (Amazon EBS) volumes and snapshots.

This article will briefly outline the important facts about Amazon EBS storage in AWS, walk you through two methods for replicating Amazon EBS volumes across AWS regions, and show you five underused Amazon EBS functions. We’ll also show you how Amazon EBS can form part of the underlying storage that powers NetApp Cloud Volumes ONTAP for AWS.

What Are AWS EBS Volumes?



Amazon Elastic Block Store (Amazon EBS) is a raw block-level storage service designed to be used with Amazon EC2 instances. When mounted to Amazon EC2 instances, Amazon EBS volumes can be used like any other raw block device: they can be formatted with a specific file system, host operating systems and applications, and have snapshots or clones made from them.

Every Amazon EBS volume that is provisioned will be automatically replicated to other storage devices in the same Availability Zone inside the AWS region to offer redundancy and high availability (guaranteed 99.999% by Amazon). AWS also offers seamless encryption of data at rest (both boot and data volumes) using Amazon-managed keys or keys customers create through Amazon Key Management Service (KMS).

Two Types of AWS EBS Volumes



There are two Amazon EBS volume type categories: SSD-backed volumes and HDD-backed volumes.

SSD-backed volumes are optimized for transactional workloads, where the volume performs a lot of small read/write operations. The performance of such volumes is measured in IOPS (input/output operations per second).

HDD-backed volumes are designed for large sequential workloads where throughput is much more important (and the performance is measured with MiB/s). Each category has two subsets.

HDD-backed Volumes Categories Chart

1. General Purpose SSD (gp2)
is balanced for price and performance, and is recommended for most use cases. General purpose SSD is a good choice for boot volumes, applications in development and testing environments, and even for low latency production apps that don’t require high IOPS. Baseline performance is directly correlated with volume size, since customers receive three IOPS per GB of volume size.

Volumes also have I/O “credits” which represent the available bandwidth that the volume can use to burst up to a higher IOPS value over a certain period of time. Credits are accumulated over time (the larger the initial volume, the more credits it will earn over time) up to a maximum of 3,000 IOPS, that can be saved for whenever the Amazon EBS volumes may need it. When a credit is depleted, the Amazon EBS volume goes back to its initial baseline performance rate. Sizes of gp2 volumes can vary from 1 GiB to 16 TiB, while maximum throughput is capped at 160 MiB/s.

Provisioned IOPS SSD (io1) are the SSD volume type to use for critical production applications and databases that need the high performance (up to 32,000 IOPS) Amazon EBS storage. Instead of credits, io1 volumes can provision a desired IOPS value, with a maximum ratio of 50:1. That means a performance rate of 5,000 IOPS can be set when provisioning a 100 GiB volume. All volumes bigger than 400 GiB can provision a maximum of 32,000 IOPS. Io1 volume sizes vary from 4 GiB to 16 TiB, while throughput is maxed at 500 MiB/s.

AWS guarantees customers that their Provisioned IOPS SSD volumes will operate at maximum 10% of loss of performance at 99.9% SLA on a yearly basis. The recommended practice is to maintain at least a 2:1 ratio between IOPS and volume size.

2. Throughput Optimized HDD (st1) is designed for applications that require larger storage and bigger throughput, such as big data or data warehousing, where IOPS is not that relevant. St1 volumes, much like SSD gp2, use a burst model, where the initial baseline throughput is tied to the volume size, and credits are accumulated over time.

Bigger volumes have higher baseline throughput and will acquire credits faster over time. Maximum throughput is capped at 500 MiB/s, sizes vary from starting 500 GiB to 16 TiB, and volumes can hold 1 TiB in credits per TiB (which is more than 30 minutes of workload under highest throughput).

Cold HDD (sc1) is a magnetic storage format suitable for scenarios where storing data at low cost is the main criteria. Sizes vary from 500 GiB to 16 TiB, throughput can reach 250 MiB/s, and a similar burst model is used as with st1 volumes, but the credits fill at a slower value (12 MiB/s opposing to 40 MiB/s with st1 volumes). Both st1 and sc1 storage types cannot be used as a boot volume, so it is recommended to use gp2 volumes for that purpose.

Copying EBS Volumes Between Regions



Since February 2017, a new Amazon EBS feature called Elastic Volumes allows AWS customers to easily switch between Amazon EBS volume types on the fly and dynamically grow volume sizes. In order to manage Amazon EBS volumes, customers can utilize AWS Management console in their web browser, or type in commands using AWS Command Line Interface (CLI).

The following two sections will demonstrate both techniques for transferring Amazon EBS volumes between different AWS regions.

Using AWS Management Console to Copy AWS EBS Volumes Between Regions



Before creating the Amazon EBS volume, login to an AWS account and select the desired region in the top right corner of the browser. Once the region is selected, click on “Services” at the top left corner of the screen, find the “Compute” area, and select “EC2.”

AWS Management Console Screenshot

In the left navigation pane, under “EC2 Dashboard,” locate the “Elastic Block Store” subsection and select “Volumes.” Once the new window displays, click the blue “Create Volume” button.

AWS Navigation Pane with Create Volume button

In the pop-up window, customize the volume (type, size, IOPS or throughput), select the desired Availability Zone, and decide whether or not to use encryption. Next, proceed to create the volume.

Pop-up window in AWS navigation pane
Once the volume is ready, attach it to the Amazon EC2 instance by right-clicking it and selecting the “Attach” option. Please note that the volume will need to be formatted inside the operating system being used.

To move the Amazon EBS volume to another region, first create a snapshot. Right-click on the volume and select “Create Snapshot.” The following screen will appear:

Snapshot Creation in AWS
Enter a name for the snapshot and, optionally, add a description. If the volume wasn’t originally encrypted, the snapshot won’t be either. By the same token, encrypted volumes will yield encrypted snapshots.

Once the snapshot is created, move it to the other region. In the Elastic Block Storage subsection in the “EC2 Dashboard” navigation menu on the left side of the screen, select “Snapshots.” Find the desired snapshot, and select “Copy.”

The following pop-up window will appear:

Pop-up for copying Snapshot creation
In the pop-up window, select “Destination Region,” as well as the option to encrypt the new snapshot. In order to utilize the new snapshot in the destination region, change the region in the top right corner of AWS Management Console.

Using AWS CLI to Copy Amazon EBS Volumes Between Regions



Before using AWS CLI, it will be necessary to configure the environment.

To create an Amazon EBS volume, use the “aws ec2 create-volume” command. Important arguments are --size, --region, --availability-zone, --volume-type, --iops (when creating SSD volumes) and --encrypted (if securing the data is important).

When the volume is created, attach it with the aws ec2 attach-volume command, and supply --volume-id and --instance-id arguments).

To move the volume to another region, first create a snapshot with “aws ec2 create-snapshot” with one mandatory argument of --volume-id. To move the snapshot, use the aws ec2 copy-snapshot command. This command requires the following parameters:

  • --source-region
  • --source-snapshot-id
  • --destination-region (optional, but necessary for cross-region copying)
  • --encrypted


All the above-mentioned commands with more examples can be found in the official documentation from Amazon.

Impacts of Amazon EBS Cross-Region Replication



Amazon snapshots are point-in-time copies of the data on the Amazon EBS volume, primarily for backup. Copying Amazon EBS snapshots from one region to another is a handy feature, both as an easy way to geographically migrate data and as a simple solution for data recovery.

However, there are two things to consider when deciding to create such copies:

  1. 1. Pricing impact: When storing volumes and snapshots in both the source and destination region, users are charged for both copies, based on the size in GiBs. It doesn’t matter that both store the same data.

  2. There are no data deduplication mechanisms, and there is also an added price for data transfers, per every GiB moved between regions.

  1. 2. Seamless synchronization impact: Data replicated across regions isn’t synced at all times. Snapshots will capture the data currently on the volume, and when moved to new region, there may be new data on the original source volume.

  2. Even scripting the entire process utilizing CLI or APIs cannot guarantee that the Amazon EBS volumes in both regions will always be in sync.

5 Functions of AWS EBS Volumes You're Not Using

1. RAID



Amazon EBS volume data is replicated across multiple servers within the same availability zone to prevent the loss of data from the failure of any single component. This replication makes Amazon EBS volumes more reliable, which means customers who follow the guidance found on the Amazon EBS and Amazon EC2 product detail pages typically achieve good performance out of the box. However, there are certain scenarios where it is necessary to achieve a higher network throughput with much better IOPS.

One way to accomplish this is by configuring a software level RAID array. RAID is supported by almost all operating systems and is used to boost the IOPS and network throughput of a volume in Amazon EBS.

Before configuring RAID, it is important to know how large your RAID array should be and how many IOPS are required. In the most basic terms, configuring a RAID array works by using two volumes as one, using the combined IOPS and network throughput of both volumes.

Things to Remember When Configuring RAID on Amazon EBS:

  • Typically, RAIDS types such as 0, 1, 5 and 6 can be configured on AWS.
  • As per AWS, the ideal RAID configuration is RAID 0 and 1: the reason for this is because in RAID 0 users can stripe across multiple volumes to gain a better distribution of I/O and therefore better performance.
  • In the same way, RAID 1 is used to achieve better availability and fault-tolerant behavior as two volumes are mirrored, with one acting as backup copy for the other in case of a failure.
  • Different types of instance resources such as General Purpose, Provisioned IOPS, Throughput Optimized, Cold HDD volumes can be combined together in a RAID 0 configuration. This uses the accumulated available bandwidth of all the volumes to achieve higher throughput and IOPS.
  • A diagrammatic representation of the architecture consisting of RAID 0 arrays should look this this:

Architecture consisting of RAID 0 arraysWhy only RAID 0? Why not other RAIDs on Amazon EBS?



AWS also supports RAID 1 on Amazon EBS volumes, which creates a greater fault tolerance capability, and makes it ideal for use cases where data durability is part of the requirement. However, it does not provide write performance improvement because the data is written to multiple volumes simultaneously.

Amazon EBS does not recommend RAID 5 and RAID 6 because the parity write operations of these RAID modes consume some of the IOPS available to your volumes. Depending on the configuration of RAID array, RAID 5 and 6 modes provide 20-30% fewer usable IOPS than a RAID 0 configuration at an increased cost. Using identical volume sizes and speeds, a two-volume RAID 0 array can outperform a four-volume RAID 6 array that costs twice as much.

It is also recommended to avoid booting from a RAID volume as the grub is typically installed on only one device and if any of the RAID array mirror device fails, you won't be able to boot the OS.

 
Click here for a step-by-step guide for configuring RAID on EBS volume.

2. Tuning Amazon EBS Performance



There are a number of factors that can affect Amazon EBS performance. However, by adhering to a few simple guidelines, you can ensure your cloud storage deployments are as optimal as possible.

  • Right-size your volumes: As Amazon EBS performance for all volumes types is directly tied to the size of the storage allocated, it is important to plan ahead for the level of performance that you will require. For example, for gp2, the size of the volume that you create will determine the baseline IOPS that you will receive, which defines the normal performance of the volume after all burst credits have been consumed.
  • Choose an appropriate block size: Data block size determines the amount of data processed in each I/O operation and is set when a filesystem is formatted. By increasing block size, it is possible to get more throughput, which can be useful for applications performing large amounts of sequential I/O, such as database systems. However, block size also determines the minimum size of a storage allocation, and so can mean using more space when working with small objects or files.
  • Optimize network bandwidth: As Amazon EBS is accessed over the network like a SAN is, it is important to have a high-performance link to the Amazon EBS volumes you are accessing from Amazon EC2. This can be achieved by either using 10 Gb networking or by using Amazon EBS-optimized Amazon EC2 instances, which provide you with a dedicated connection between your Amazon EC2 instances and the Amazon EBS storage volumes you have allocated.

3. Increasing Amazon EBS Volume Size While in Use



Amazon EBS is elastic, which means you can modify the size, IOPS, and the types of volume on the go without difficulty.

To modify the instance size, it is required to stop the instance, take snapshot of the volume, create a new volume, and attach it.

Recently, Amazon announced that you can now modify Amazon EBS volume size and type while they are in an in-use state, significantly decreasing the effort required to extend or modify attributes of a volume.

Executing this modification of volume size comes at no additional charge, although there is a charge for higher storage. Increasing Amazon EBS volume size also helps in production use cases where there is an application with a database on Amazon EBS.

In the past, DB size would increase, but the administrator would not be able to modify the size as that requires some down time which is not possible in a 24 x 7 live system. Now with this feature, it’s easier to modify the size of a volume. It is important to note that this feature is not available for previous generation instance types.

In general, the following steps are involved in modifying a volume:

  1. Issue the modification command.
  2. Monitor the progress of the modification.
  3. If the size of the volume was modified, extend the volume's file system to take advantage of the increased storage capacity.

For more visit: Modifying the Size, IOPS, or Type of an Amazon EBS Volume.

4. Amazon CloudWatch Events for Amazon EBS



AWS automatically provides data such as instance metrics and volume status checks via Amazon CloudWatch. These are state notifications and can be used to monitor Amazon EBS volumes.

Typically, monitoring data is available on Amazon CloudWatch in five-minute periods at no additional charge. Amazon CloudWatch also provides a monitoring option with for PIOPS (Provisioned IOPS) instance types such as io1; data for such instance types are available on Amazon CloudWatch at every one minute, accessible via Amazon CloudWatch API or the Amazon EC2 console, which is part of the AWS management console.

Amazon EBS has several state notification events for volumes:

  • Ok: Normal volume performance
  • Warning: Degraded volume performance
  • Impaired: Stalled volume performance, severely impacted
  • Insufficient data: No information is available on the volume


A user can always configure Amazon CloudWatch alarms to trigger an SNS notification based on a state change. In addition to this, Amazon CloudWatch Events allow a user to configure Amazon EBS to emit notifications whenever a snapshot or encryption state has a status change. Users can also use Amazon CloudWatch Events to establish rules that trigger programmatic actions in response to changes in the snapshot or encryption key states.

In a use case scenario, Customer XYZ wants to ensure whenever a snapshot is created, the snapshot is shared with another account or copied to another region for disaster-recovery purposes. A possible solution is for Customer XYZ to leverage Amazon CloudWatch to establish rules that trigger programmatic actions in response to a change in snapshot or encryption.

Amazon CloudWatch’s CopySnapshot feature, the event is sent to Customer XYZ’s AWS account when an action to copy a snapshot completes. If either event succeeded then it should trigger an Amazon Lambda function which will automatically copy the snapshot from one specified region to another.

Refer following documentation to see a step-by-step guide for implementing triggers on Amazon CloudWatch Events feature: Create Triggers on CloudWatch Events

5. Sharing Snapshots Across AWS Accounts



With Amazon EBS, users can share snapshots publicly and privately. Sharing snapshots publicly means that all the data in the snapshot is shared with all AWS users, hence it is advisable that proper checks be performed before changing permissions of the snapshot.

AWS allows you to share unencrypted volume snapshots privately and publicly. However, encrypted snapshots cannot be shared with all AWS users, only with an approved list of AWS users using custom CMK (customer managed key).

Encrypted snapshots help achieve maximum security between sharer and receiver. Snapshots are encrypted with CMK. To share the snapshot with others, the same custom CMK which was used to encrypt volume is needed.

There are several things to consider before sharing snapshots. Below are some of snapshot sharing’s limitations:

  • You can share both encrypted and unencrypted volume snapshots between different AWS accounts.
  • Encrypted snapshots can only be shared if a custom key has been created and shared with another account. For more visit: How to Copy an Encrypted Volume to Another Account.


A step-by-step guide for snapshot creation can be found here:

How to Create an AWS EBS Snapshot

AWS EBS and Cloud Volumes ONTAP (formerly ONTAP Cloud)



NetApp’s Cloud Volumes ONTAP enables companies to leverage enterprise storage capabilities in cloud environments. By deploying Cloud Volumes ONTAP instances in desired regions and utilizing NetApp’s SnapMirror technology, customers can take snapshots of their data without requiring additional storage or impacting an application's performance, and then replicate those snapshots to another region while keeping them synchronized automatically per a user-defined schedule.

If needed, Cloud Volumes ONTAP can also leverage physical on-premises ONTAP systems as a source or destination, allowing users to backup or restore data to and from the AWS cloud. Additionally, SnapMirror only needs to create a single baseline copy once: subsequent copies after the baseline only replicate the changed data.

Other benefits for using Cloud Volumes ONTAP in AWS is because of NetApp’s cost-saving storage efficiencies such as thin provisioning, data deduplication and compression. These features allow Cloud Volumes ONTAP to consume much less underlying Amazon EBS capacity than otherwise possible on AWS. These storage efficiencies also persist for the SnapMirror relationships, meaning that the data is replicated in a space efficient manner, which saves you on data egress charges. Another of these features, data tiering, allows AWS users to store infrequently-used data, DR environments, and snapshot data on Amazon S3 until that data is needed, when it is automatically brought back up to performant, low-latency Amazon EBS volumes.

Conclusion



Amazon EBS is one of the most important storage formats available in the public cloud today, so it is important to make sure you know all the ins and outs of this versatile AWS offering. In this article, we’ve looked at a number of ways that you can manage, configure, and augment your Amazon EBS volumes, including how to get better performance, higher levels of protection, and optimized costs with Cloud Volumes ONTAP for AWS.  

To try NetApp Cloud Volumes ONTAP for AWS storage today, sign up for a free 30-day trial today.

AWS Storage Options
-