NetApp Cloud Volumes Service
for AWS: Benchmarks

File Services Workloads - IOPS

Using the FIO load generator and multiple EC2 instances, we found a single volume capable of delivering over 460K 4KiB reads and  125K 4KiB writes per second. For a single instance NFS I/O limits, with only a very little bit of NFS tuning (see nconnect mount options), we were able to drive over 200K 4KiB reads and the same 120K 4KiB writes. NetApp is working with other distributions of Linux to see the nconnect feature (present today in SLES15 and the 5.3 Linux Kernel) adopted more generally. Without the nconnect feature, testing has shown a single client capable of driving ~80K 4KiB reads/s or writes/s.

Read the full baseline performance story here.

File Services Workloads - Throughput

Our sequential tests were done using the exact same tests and procedures. In this case, again using the nconnect mount option (nconnect=16) and a single client we saw ~1,700MiB/s of sequential reads and 570MiB/s of sequential writes. The former is possible because nconnect creates multiple network connections (flows) for the single storage endpoint thereby getting around the AWS 5Gbps per network flow limit. The latter 570MiB/s of writes is due to AWS instance level restrictions imposed upon egress over direct connect. From a multi-client configuration, ~4500MiB/s of sequential reads and ~1500MiB/s of sequential writes were observed. For sequential workloads, a 64KiB operation size as well as a 64K rsize and wsize were selected.

Oracle IOPS with EBS and Cloud Volumes Service

The graphs compare the performance when using EBS and when using NetApp Cloud Volumes Service for AWS. The graph on the right shows that Oracle is able to drive 250,000 file system IOPS at 2ms when using the c5.18xlarge instance and a single volume provisioned from the Cloud Volumes Service, or 144,000 file system operations at below 2ms using the c5.9xlarge.

Oracle IOPS at various read/write ratios

The graph to the left provides more performance examples of how Oracle workloads behave on Cloud Volumes Service for AWS when run on the same c5.18xlarge EC2 instance shown above. Note that Cloud Volumes Service provides a high level of IOPS at 2 ms or below across a range of read/write ratios. Also note that in all cases adding a second volume increases the IOPS provided.

For more information on Oracle Performance and Storage Comparison in AWS: Cloud Volumes Service, EBS, EFS.

Hadoop Spark gets 3,100MB/s

The graph on the right shows performance when using Amazon S3 and NetApp Cloud Volumes Service for AWS (service levels Standard and Premium). It shows that Spark is able to achieve an average throughput of 3,100MB/s against a single Cloud Volumes Service volume when run on 15 C5.9xlarge Amazon EC2 instances.

Although the price of the Premium service level ($0.20/GB/month) is higher than both the Standard service level ($0.10/GB/month) and the upfront costs of Amazon S3 (capacity + egress), the increased bandwidth results in both an overall cost reduction and improved run time, making the Premium service level more cost-efficient overall.

API costs make up a large portion of the Amazon S3 price. GET requests for Standard Access Tier are priced at $0.0004 per 1,000, so the cost of continuously using Amazon S3 for primary analytics clusters can add up to ~$170,000 annually.

Read this in-depth blog on Spark performance using Cloud Volumes Service for AWS.

MySQL gets 25,000 IOPS

The graph to the right shows the performance of a MySQL workload running on c5.18xlarge. Run against the single Cloud Volumes Service instance, we were able to generate close to 25,000 IOPS at 4 ms latency, and 22,000 IOPS at 3 ms latency.

I/O Comparison of Cloud Volumes Service and AWS EFS

The graphs demonstrate how Cloud Volumes Service for AWS and EFS compare when running random and sequential workloads.

The following are the instance and limits in the test:

Instance limits:

Elastic File System: Maximum 250/MB/s per instance throughput
Cloud Volumes: 1GB/s maximum per instance throughput (512MB/s read + 512MB/s write)

Limits:

Elastic File System: Maximum 7,000 IOPS per volume (as documented by AWS)
Cloud Volumes: ~200,000 maximum IOPS per volume as tested

AWS Regional Performance

Here are the observed latencies in milliseconds between Amazon EC2 instances and Cloud Volumes Service for AWS based on the regions that the service is available in today.

Genomics HPC workload on Cloud Volumes Service for AWS - Sequential Reads

Cloud Volumes Service for AWS was tested against competing products to move a database designed for genomic workloads to the cloud. The sequential read benchmark had 1 hour to complete - with a goal of ~10 TiB/hr (2,900MiBps). The test itself comprised 2,500 files representing 2000TIB of content.

The throughput achieved using Cloud Volumes Service volume is equal to 2,887MiBps or 9.91TiB/hr, which is 2.1x the rate of the four self-managed NFS servers and 3x times that of the Provisioned Throughput configured EFS volumes. Cloud Volumes Service for AWS achieved the results while also providing snapshot copies, which the other options were not able to provide, or not able to provide without impacting performance.

While the chart indicates a throughput of 2,887 MiBps the test data shows that only a handful of workers took longer than the rest of the 2,500 workers. In fact, most of the workers achieved a throughput of roughly 3,500 MiBps.

Genomics HPC workload on Cloud Volumes Service for AWS - Random Queries

As additional data points, the graph on the left shows the results of the second use case - that of a SQL-type query of the 2,500 genomic files. A lower time to completion indicates strong performance. Cloud Volumes Service was able to access data from 100,000 individuals in less than an hour, while also providing snapshot copies like in the previous case.

Read this blog and accompanying report to learn more about this test and the benefits the genomics company gets with Cloud Volumes Service.