AWS EFS

AWS EFS: Is It the Right Storage Solution for You?

[Cloud Volumes ONTAP, File Services, AWS, Elementary, 9 minute read, AWS EFS, A]

Amazon EFS is an AWS file sharing service that lets you manage file shares, like those used on traditional networks, and mount them on cloud or on-premises machines using the NFS protocol.

In this post, we’ll review the basics of AWS EFS, give you some tips to help you optimize your performance, and see how it compares to other AWS storage options. Finally, we’ll show you how NetApp Cloud Volumes ONTAP can help you get the most out of whatever file service option you choose.

In this article, you will learn:

What Is AWS EFS?

AWS Elastic File System (EFS) is one of three main storage services offered by Amazon. It is a scalable, cloud-based file system for Linux-based applications and workloads that can be used in combination with AWS cloud services and on-premise resources. EFS offers a choice between two storage classes, Infrequent Access and Standard access, depending on your needs. Standard access storage is designed for frequently accessed files while Infrequent Access is intended for storing long-lived but less used files at a lower cost.

EFS uses the NFSv4 protocol for its file system structure, which mirrors a standard on-premise structure and simplifies transferring and accessing your files. It can be used in combination with Elastic Cloud Compute (EC2) instances or as a stand-alone file system. EFS does not require storage provisioning and is pay-for-use allowing you to scale services as needed.

Some of EFS’ key features include:

Shared Storage
Files are accessible from AWS services as well as from on-premise. They can be simultaneously accessed by up to a thousand EC2 instances within the cloud or via VPN or AWS Direct Connect, making EFS good for hybrid solutions. File access is possible across multiple AWS Availability Zones (AZs) and Regions, for easier collaboration and global remote work.

Scalable Performance
EFS is designed for low latency with IOPS and throughput that scale with usage and the number of attached instances, meaning that as storage size grows, performance increases. At peak performance, it offers 10 GB/sec throughput and 500k IOPS. EFS scales automatically as data is moved in or out, minimizing fears of running out of space or paying for storage you aren’t using. Learn more about how file sharing impacts cloud scalability and agility.

Secure and Compliant
EFS allows multiple layers of security and relies on your existing security infrastructure. It can be used with Amazon's Identity and Access Management (IAM) roles as well as VPC security groups and allows you to define individual file permissions using POSIX. EFS has built-in compliance with common regulatory standards, including PCI DSS, HIPAA, and SOC with the ability to meet others if necessary. Learn more about best practices for secure cloud file sharing.

AWS EFS Backup Solutions

Since a storage solution is only really useful when it can be backed up, you should be aware of your backup options with EFS. Keep in mind, other ways exist if you want to use third-party products integrated with AWS. Also, learn more about general considerations for backup and archiving in the cloud.

Amazon Backup Service
AWS Backup is a fully managed service that allows you to create, manage, and automate incremental backups according to a schedule you define through a central location. This system is PCI and ISO compliant and HIPAA eligible, to ensure that your compliance needs are covered. It is possible to use AWS Backup whether your system has a cloud-native, hybrid, or on-premise configuration.

This solution is easy to implement, and incremental backups help keep your costs low but requires manually pausing the applications and processes being backed up and only allows backups to be stored on EFS.

EFS to EFS Backup
There is no built-in EFS backup, and EFS does not have a native snapshot mechanism. So, before AWS Backup was released, backups had to be done using a template in AWS CloudFormation. This involves using scripts to access the AWS Data Pipeline, from which you must transfer data between multiple services before finally storing the backed-up data in EFS. With this process, you are still able to control backup schedules and life cycles as with the AWS Backup Service.

The main downsides of this option are that it does require some programming knowledge and is not easily monitored. Additionally, if you are not careful to change time constraints according to the amount of data you are backing up, your process can fail.

Backing Up to Amazon S3
Backing up your data to S3 is another option you might consider and one that can help decrease your storage costs. This process begins the same as the EFS to EFS backup but rather than moving the incremental backups from S3 to EFS at the end of the process, they are simply left in S3.

Like EFS to EFS, you can schedule your backups and manage life cycle length but you must also have some programming ability and be familiar with the use of scripts for automating processes.

How to Optimize AWS EFS Performance

To get the greatest ROI with EFS, you will need to take steps to optimize your configuration performance. Although specific changes will depend on your needs, the following tips apply to most EFS users.

Monitor Your EFS Burst Credits
One of the biggest shocks for EFS adopters comes the first time they run out of burst credits and they see their performance sharply drop. To avoid this, make sure that you monitor your burst credits either manually or by adding alerts in CloudWatch that will notify you when they run low. Making sure to schedule backups during periods of low use or during non-working hours will help save your credits for when they are truly needed.

You should keep in mind that EFS volumes start with a .5MB/s transfer rate and only 7.2 minutes of burst credits for throughput up to 100MB/s, which is probably not enough to meet your needs. You can increase these numbers by increasing the amount of data you’re storing but there is no way to simply buy additional credits.

Don’t Run Applications from EFS
If you plan to run and host applications from EFS, Elastic Block Storage (EBS) is a better option for you. EFS is not meant for the large file read volumes or fast speed that tasks like managing codebases or application deployment require, so you should not try to use it for these tasks. Instead, stick to tools that deploy code to local filesystems or containers.

Using EFS as it is designed, for storage of media assets, exported data files, asynchronous logs, etc. will ensure that you see the sort of performance you’re expecting and that your productivity doesn’t suffer.

Choose the Right Performance Mode
EFS is designed with two different performance mode options: General Purpose and Max I/O.

General Purpose is the default setting and the one that will work best for most users. It focuses on lower throughput levels in exchange for lower latency and is good for tasks like web-hosting and content management.

Max I/O focuses on higher throughput at the cost of higher latency, making it better for tasks such as big data analysis or media processing.

Keep in mind, both will see an increase in throughput performance as you attach more concurrent instances as long as the workload can be spread across the instances.

Amazon Storage Services: EFS vs EBS vs S3

To know whether EFS truly is the best option for you, you’ll want to compare it to your other two main options, EBS and S3.

AWS EFS
EFS is fairly easy to learn to use because it mirrors a traditional file hierarchy structure. Since it allows multiple instances to be connected simultaneously across regions and AZs, it does not require as much redundant storage as other options and allows for easy cloud file sharing and collaboration.

As mentioned, EFS is ideal for use in global content management systems, big data analytics, and media processing workflows.

AWS EBS
EBS uses volumes to provide low-latency block storage for use with EC2 instances. It can be a bit complicated to set up as it requires that you choose your volume type, according to performance requirements, from the start and it doesn’t automatically scale like other options.

EBS is designed for application hosting and storage and high-volume databases. Learn more about AWS EBS in our extensive series of blog posts.

AWS S3
Simple Storage Service (S3) provides object storage, without the use of EC2 instances, that is accessible directly through the internet. It is the cheapest option and provides a lot of flexibility in storage, but can require some programming proficiency as it is typically managed through AWS Software Development Kits (SDKs). Learn how to mount Amazon S3 as a drive to simplify management.

S3 works well for static website hosting, version management, log file storage, media distribution or streaming, and big data analytics.

For a side-by-side comparison of these three options:

 

EFS

EBS

S3

Cost*

$0.3 per GB/month and $6 per MB/month for provisioned throughput

$.045-$0.125 per GB/month, depending on volume type

$0.021-0.023 per GB/month depending on total storage

Performance

Up to 500k IOPS and 10+ GB/s throughput

 

Minimum of 3GB/s

Varies by volume type

 

Minimum of 3 IOPS per GB of storage

3500 PUT/LIST/DELETE requests per second

 

5500 GET requests per second

Availability and Accessibility

Highly available but no SLA defining percent

 

Accessed by up to 1K instances from multiple AZ or Regions

99.99% available

 

Accessed via a single EC2 instance

99.99% available

 

Accessed via Internet and API

Access Control

Managed via user and group level permissions

File access requires endpoint access

Managed via IAM and security groups

Managed via IAM and bucket policies

Storage and File Limits

Unlimited storage size

47.9 TB file size

16 TB storage per volume with unlimited volumes

File size limited by volume size

Unlimited storage size

5TB file size limit

*AWS pricing is rather complex but the NetApp AWS calculator can help you better understand and reduce your costs.

How Cloud Volumes ONTAP Can Help You Support File Shares

NetApp Cloud Volumes ONTAP, the leading enterprise-grade storage management solution, delivers secure, proven storage management services on AWS, Azure and Google Cloud. Cloud Volumes ONTAP supports up to a capacity of 368TB, and supports various use cases such as file services, databases, DevOps or any other enterprise workload.

Cloud Volumes ONTAP provides a cloud file share service that supports efficient NFS and SMB file storage and facilitates secure cloud file sharing.

Want to learn more about AWS EFS? Read our articles:

AWS File Sharing Services with Amazon EFS

AWS EFS is Amazon’s fully-managed cloud file sharing solution which supports the NFS protocol. Before selecting a file service it’s important to learn about the available options to see which solution can best meet your requirements.

In this article we take a deep dive into AWS EFS, examining its capabilities, storage capacity, costs and limitations. We take our analysis deeper by comparing the file service to Cloud Volumes ONTAP, another file sharing solution that comes with built-in storage efficiencies, cloning, tiering to Amazon S3 and cost-efficient snapshots.   

Read “AWS File Sharing with AWS EFS” here. 

EFS Backup: Backing Up NFS Data in AWS

AWS EFS, Amazon’s file sharing service, does not provide built-in backup for NFS file share data. For this reason, AWS EFS customers must ensure they have a solid backup plan in place.   

This article examines different options for backing up AWS EFS data. These include EFS to EFS backup, which can tend to be complicated to execute, EFS to Amazon S3 and the open-source tool Terraform. Although Terraform can more flexibly adapt to suit requirements, it requires extensive expertise to implement. Cloud Volumes ONTAP, NetApp’s file sharing service, is an alternative to AWS EFS which supports and backs up NFS, as well as SMB and CIFS data.  

Read “EFS Backup: Backing Up NFS Data in AWS” here. 

 

EFS Performance Do’s and Don’ts

AWS EFS, Amazon’s file sharing service, is a great service - assuming you know how and when to make use of it.

This article explains when it’s best to use EFS, such as in the case of big data or media processing, and when EFS is not the optimal choice, for instance with workloads that require snapshots, applications that need single file access, and OLTP databases. It will give you tips for how to optimize EFS performance, determining which performance and throughput modes to use, what to monitor, how to handle backup and other important settings to be aware of. Finally, we’ll demonstrate how NetApp’s Cloud Volumes ONTAP can help to further optimize performance. 

Read “EFS Performance Do’s and Don’ts” here.       

AWS NFS File Shares with Amazon EFS: 5 Key Considerations

Network File System (NFS) is a distributed file system protocol that allows files to be accessed over a network. AWS EFS is an Amazon cloud solution with NFS capabilities.

In this article we look at the history of NFS and explore 5 major issues you should be aware of when using AWS EFS for your NFS workloads. This includes how to manage backup, operating system support, on-premises access to files, and leveraging availability zones. We’ll also show how NetApp’s Cloud Volumes ONTAP solution enhances NFS and helps address these considerations.  

Read “AWS NFS File Shares with Amazon EFS: 5 Key Considerations” here. 

EFS IA Vs. Cloud Volumes ONTAP Data Tiering: File Share Tiering Options on AWS

Across industries, most companies have a 80/20 split between the amount of data that is inactive over active. The same is true with file share data stored on services such as AWS EFS. To reduce the cost of storing this large amount of infrequently accessed files, AWS has created a new storage class for AWS EFS, called EFS Infrequent Access (EFS IA).

EFS IA effectively acts as a form of AWS tiered storage. What are the benefits of tiering file share data from EFS Standard to EFS IA? And how does this new solution from AWS compare to keeping file share data on Amazon EBS with Cloud Volumes, which leverages automated data tiering to Amazon S3? Find out in this post.

Read "EFS IA Vs. Cloud Volumes ONTAP Data Tiering: File Share Tiering Options on AWS" here.

AWS Shared Storage for Files, Block & Object Storage 

Shared storage enables you to store, distribute, and share your data across multiple locations and users. AWS offers a wide range of shared options for storage, including shared files, block, and object storage.

This article explains how shared storage works on AWS, and the main differences between AWS shared storage options.

Read more: AWS Shared Storage for Files, Block & Object Storage

New call-to-action

-