Blog

AWS EFS: Is It the Right Storage Solution for You?

Amazon EFS is an AWS file sharing service that lets you manage file shares, like those used on traditional networks, and mount them on cloud or on-premises machines using the NFS protocol.

In this post, we’ll review the basics of AWS EFS, give you some tips to help you optimize your performance, and see how it compares to other AWS storage options. Finally, we’ll show you how NetApp Cloud Volumes ONTAP can help you get the most out of whatever file service option you choose.

In this article, you will learn:

What Is AWS EFS?

AWS Elastic File System (EFS) is one of three main storage services offered by Amazon. It is a scalable, cloud-based file system for Linux-based applications and workloads that can be used in combination with AWS cloud services and on-premise resources. EFS offers a choice between two storage classes, Infrequent Access and Standard access, depending on your needs. Standard access storage is designed for frequently accessed files while Infrequent Access is intended for storing long-lived but less used files at a lower cost.

EFS uses the NFSv4 protocol for its file system structure, which mirrors a standard on-premise structure and simplifies transferring and accessing your files. It can be used in combination with Elastic Cloud Compute (EC2) instances or as a stand-alone file system. EFS does not require storage provisioning and is pay-for-use allowing you to scale services as needed.

Some of EFS’ key features include:

Shared Storage
Files are accessible from AWS services as well as from on-premise. They can be simultaneously accessed by up to a thousand EC2 instances within the cloud or via VPN or AWS Direct Connect, making EFS good for hybrid solutions. File access is possible across multiple AWS Availability Zones (AZs) and Regions, for easier collaboration and global remote work.

Scalable Performance
EFS is designed for low latency with IOPS and throughput that scale with usage and the number of attached instances, meaning that as storage size grows, performance increases. At peak performance, it offers 10 GB/sec throughput and 500k IOPS. EFS scales automatically as data is moved in or out, minimizing fears of running out of space or paying for storage you aren’t using. Learn more about how file sharing impacts cloud scalability and agility.

Secure and Compliant
EFS allows multiple layers of security and relies on your existing security infrastructure. It can be used with Amazon's Identity and Access Management (IAM) roles as well as VPC security groups and allows you to define individual file permissions using POSIX. EFS has built-in compliance with common regulatory standards, including PCI DSS, HIPAA, and SOC with the ability to meet others if necessary. Learn more about best practices for secure cloud file sharing.

AWS EFS Backup Solutions

Since a storage solution is only really useful when it can be backed up, you should be aware of your options with EFS. Keep in mind, other ways exist if you want to use third-party products integrated with AWS. Also, learn more about general considerations for backup and archiving in the cloud.

Amazon Backup Service
AWS Backup is a fully managed service that allows you to create, manage, and automate incremental backups according to a schedule you define through a central location. This system is PCI and ISO compliant and HIPAA eligible, to ensure that your compliance needs are covered. It is possible to use AWS Backup whether your system has a cloud-native, hybrid, or on-premise configuration.

This solution is easy to implement, and incremental backups help keep your costs low but requires manually pausing the applications and processes being backed up and only allows backups to be stored on EFS.

EFS to EFS Backup
There is no built-in EFS backup, and EFS does not have a native snapshot mechanism. So, before AWS Backup was released, backups had to be done using a template in AWS CloudFormation. This involves using scripts to access the AWS Data Pipeline, from which you must transfer data between multiple services before finally storing the backed-up data in EFS. With this process, you are still able to control backup schedules and life cycles as with the AWS Backup Service.

The main downsides of this option are that it does require some programming knowledge and is not easily monitored. Additionally, if you are not careful to change time constraints according to the amount of data you are backing up, your process can fail.

Backing Up to Amazon S3
Backing up your data to S3 is another option you might consider and one that can help decrease your storage costs. This process begins the same as the EFS to EFS backup but rather than moving the incremental backups from S3 to EFS at the end of the process, they are simply left in S3.

Like EFS to EFS, you can schedule your backups and manage life cycle length but you must also have some programming ability and be familiar with the use of scripts for automating processes.

How to Optimize AWS EFS Performance

To get the greatest ROI with EFS, you will need to take steps to optimize your configuration performance. Although specific changes will depend on your needs, the following tips apply to most EFS users.

Monitor Your EFS Burst Credits
One of the biggest shocks for EFS adopters comes the first time they run out of burst credits and they see their performance sharply drop. To avoid this, make sure that you monitor your burst credits either manually or by adding alerts in CloudWatch that will notify you when they run low. Making sure to schedule backups during periods of low use or during non-working hours will help save your credits for when they are truly needed.

You should keep in mind that EFS volumes start with a .5MB/s transfer rate and only 7.2 minutes of burst credits for throughput up to 100MB/s, which is probably not enough to meet your needs. You can increase these numbers by increasing the amount of data you’re storing but there is no way to simply buy additional credits.

Don’t Run Applications from EFS
If you plan to run and host applications from EFS, Elastic Block Storage (EBS) is a better option for you. EFS is not meant for the large file read volumes or fast speed that tasks like managing codebases or application deployment require, so you should not try to use it for these tasks. Instead, stick to tools that deploy code to local filesystems or containers.

Using EFS as it is designed, for storage of media assets, exported data files, asynchronous logs, etc. will ensure that you see the sort of performance you’re expecting and that your productivity doesn’t suffer.

Choose the Right Performance Mode
EFS is designed with two different performance mode options: General Purpose and Max I/O.

General Purpose is the default setting and the one that will work best for most users. It focuses on lower throughput levels in exchange for lower latency and is good for tasks like web-hosting and content management.

Max I/O focuses on higher throughput at the cost of higher latency, making it better for tasks such as big data analysis or media processing.

Keep in mind, both will see an increase in throughput performance as you attach more concurrent instances as long as the workload can be spread across the instances.

Amazon Storage Services: EFS vs EBS vs S3

To know whether EFS truly is the best option for you, you’ll want to compare it to your other two main options, EBS and S3.

AWS EFS
EFS is fairly easy to learn to use because it mirrors a traditional file hierarchy structure. Since it allows multiple instances to be connected simultaneously across regions and AZs, it does not require as much redundant storage as other options and allows for easy cloud file sharing and collaboration.

As mentioned, EFS is ideal for use in global content management systems, big data analytics, and media processing workflows.

AWS EBS
EBS uses volumes to provide low-latency block storage for use with EC2 instances. It can be a bit complicated to set up as it requires that you choose your volume type, according to performance requirements, from the start and it doesn’t automatically scale like other options.

EBS is designed for application hosting and storage and high-volume databases. Learn more about AWS EBS in our extensive series of blog posts.

AWS S3
Simple Storage Service (S3) provides object storage, without the use of EC2 instances, that is accessible directly through the internet. It is the cheapest option and provides a lot of flexibility in storage, but can require some programming proficiency as it is typically managed through AWS Software Development Kits (SDKs). Learn how to mount Amazon S3 as a drive to simplify management.

S3 works well for static website hosting, version management, log file storage, media distribution or streaming, and big data analytics.

For a side-by-side comparison of these three options:

 

EFS

EBS

S3

Cost*

$0.3 per GB/month and $6 per MB/month for provisioned throughput

$.045-$0.125 per GB/month, depending on volume type

$0.021-0.023 per GB/month depending on total storage

Performance

Up to 500k IOPS and 10+ GB/s throughput

 

Minimum of 3GB/s

Varies by volume type

 

Minimum of 3 IOPS per GB of storage

3500 PUT/LIST/DELETE requests per second

 

5500 GET requests per second

Availability and Accessibility

Highly available but no SLA defining percent

 

Accessed by up to 1K instances from multiple AZ or Regions

99.99% available

 

Accessed via a single EC2 instance

99.99% available

 

Accessed via Internet and API

Access Control

Managed via user and group level permissions

File access requires endpoint access

Managed via IAM and security groups

Managed via IAM and bucket policies

Storage and File Limits

Unlimited storage size

47.9 TB file size

16 TB storage per volume with unlimited volumes

File size limited by volume size

Unlimited storage size

5TB file size limit

*AWS pricing is rather complex but the NetApp AWS calculator can help you better understand and reduce your costs.

How Cloud Volumes ONTAP Can Help You Support File Shares

NetApp Cloud Volumes ONTAP, the leading enterprise-grade storage management solution, delivers secure, proven storage management services on AWS, Azure and Google Cloud. Cloud Volumes ONTAP supports up to a capacity of 368TB, and use cases such as file services, databases, DevOps or any other enterprise workload.

Cloud Volumes ONTAP provides a cloud file share service that supports efficient NFS and SMB file storage and facilitates secure cloud file sharing.

Want to learn more? Check out these blogs:

Want to get started? Try out Cloud Volumes ONTAP today with a 30-day free trial.

-