More about AWS Database
- MongoDB on AWS: Managed Service vs. Self-Managed
- Deployment Options on AWS for Apache Cassandra
- AWS PostgreSQL: Managed or Self-Managed?
- MySQL Database Migration: Amazon EC2-Hosted vs. Amazon RDS
- AWS MySQL: MySQL as a Service vs. Self Managed in the Cloud
- AWS ElastiCache for Redis: How to Use the AWS Redis Service
- AWS Data Analytics: Choosing the Best Option for You
- Database Case Studies with Cloud Volumes ONTAP
- Amazon DocumentDB: Basics and Best Practices
- AWS NoSQL: Choosing the Best Option for You
- AWS Big Data: 6 Options You Should Consider
- Oracle on AWS: Managed Service vs. Managed Storage Options
- AWS Database as a Service: 8 Ways to Manage DBs in AWS
- AWS Oracle RDS: Running Your First Oracle Database on Amazon
- SQL Server in AWS: Two Deployment Options
- DynamoDB Pricing: How to Optimize Usage and Reduce Costs
- AWS Databases: The Power of Purpose-Built Database Engines
- AWS Oracle: How to Lift and Shift Your Oracle DB to Amazon
- Overcome AWS RDS instance Size Limits with Data Tiering to AWS S3
Of Amazon’s 212 cloud computing services, fifteen are purpose-built database engines. In this post, we focus on eight AWS database services: Amazon RDS, Amazon Aurora, Amazon DynamoDB, Amazon DocumentDB, Amazon ElastiCache, Amazon Neptune, Amazon Timestream, and Amazon Quantum Ledger Database (QLDB).
We offer a review of the functions, feature, capabilities and use cases of each of these services, to help you choose the database that is a perfect match for your use case and project. In addition, we’ll show how NetApp Cloud Volumes ONTAP can help solve AWS database migration and management challenges.
This is part of our series of comprehensive guides on cloud storage technology.
In this article, you will learn:
- Types of databases on AWS
- AWS database services
- AWS migration services
- AWS database management with Cloud Volumes ONTAP
Types of Database Services Offered on AWS
AWS offers a wide range of database services for you to choose from. The service fall into two groups: relational and non-relational (NoSQL).
AWS Relational Database Services
Relational databases store data in tabular form with columns and rows, and can be queried using the SQL query language. In these databases, columns represent attributes and rows represent records. Each field in the table is a data value.
Use cases for relational databases on Amazon include:
- Enterprise resource planning (ERP) apps
- Customer relationship management (CRM) apps
- Finance data
- Data warehousing
The primary Amazon services providing relational databases are:
- Amazon Aurora
- Amazon RDS
- Amazon Redshift
AWS NoSQL Database Services
Relational databases are not suitable for many use cases, especially those requiring very high performance or dynamic scalability. NoSQL, or non-relational databases, break the paradigm of storing data in tables with columns and rows, allowing them to distribute and process data more efficiently. NoSQL is commonly used to handle big data - large volumes of unstructured or semi-structured data.
The table below shows the main NoSQL databases services offered by AWS.
Type of Database
Key-value databases store data as a collection of key-value pairs with the key as an ID. These databases can store various types of data, including simple and compound objects.
● Real-time bidding
● eCommerce shopping carts
● Product catalogs
● Customer preferences
● Amazon DynamoDB
Document databases store data in JSON or JSON-like documents. You can query data using the same document-model format used in programming applications.
● Content management systems
● Customer profiles and personalization
● Mobile apps
● Amazon DocumentDB
In-memory databases store data in-memory for low-latency access. You can use these stores as a database, cache, message broker, or queue.
● Session stores
● Geospatial services
● Pub/sub messaging
● Real-time streaming
● Amazon ElastiCache for Memcached
● Amazon ElastiCache for Redis
Graph databases are a type of NoSQL (non-relational) database. This database type represents relationships directly. You can query data with specific graph languages.
● Fraud detection
● Social networking
● Recommendation engines
● Knowledge graphs
● Data lineage
● Amazon Neptune
Time-series databases store data in time-order and as append-only. You can query data over various time intervals.
● Application monitoring
● Industrial telemetry
● IoT applications
● Amazon Timestream
Ledger databases store data in an immutable, transparent, and cryptographically verifiable log. This log is owned by a trusted central authority to ensure provenance.
● Insurance claims
● HR and payroll
● Retail inventories
●Amazon Quantum Ledger Database
Note: While we listed in-memory databases under the non-relational category, there are in-memory databases that are relational, SQL databases. A notable example is SAP HANA. You can run SAP HANA on AWS using the managed SAP HANA instance.
AWS Databases Services
Once you understand what options are available to you for databases in AWS, you can begin narrowing them down. The following services are some of the most commonly used. Keep in mind when reviewing these descriptions that frequently, AWS customers implement multiple database types to meet their needs. You too should consider multiple options if one doesn’t meet all of your needs.
Amazon RDS is a managed, relational database service that includes six different database options. These include AWS Oracle, PostgreSQL, AWS MySQL, MariaDB, SQL Server, and Amazon Aurora. You can manage these database engines from a centralized management console, a command-line interface, or via API calls. When using this service, many administrative tasks are automated, including database setup, hardware provisioning, backup, and updating.
Use cases of Amazon RDS include:
- Web and mobile applications—provides the scalability, availability, and throughput needed for enterprise-grade applications.
- eCommerce applications—provides flexibility, security, and PCI compliance needed for eCommerce.
- Mobile and online games—provides high-throughput and availability to ensure that games remain online and responsive to players.
Amazon Aurora is a fully managed relational database engine designed specifically for AWS. It is MySQL and PostgreSQL compatible with minor changes to your source database. Aurora includes features for self-healing, fault tolerance, point-in-time recovery, and continuous backup.
Use cases for Amazon Aurora include:
- Enterprise applications—including customer relationship management and enterprise resource planning solutions.
- Software as a Service (SaaS) offerings—including those requiring significant storage and compute scalability.
- Web and mobile gaming applications—including those requiring massive storage, high throughput, and high-availability.
Amazon DynamoDB is a fully managed, document and key-value database. It includes features for multi-master, multi-region used along with built-in security, automated backup and restoration, and in-memory caching. DynamoDB can provide support for serverless web apps, microservices, and mobile backends.
Use cases of Amazon DynamoDB include:
- Ad tech—including clickstreams, user events, and user profiles.
- Gaming—including leaderboards, player data stores, and game states.
- Retail—including online shopping carts, inventory tracking, and customer profiles.
- Banking and finance—including event-driven transaction processing, fraud detection, and change data capture.
- Media and entertainment—including digital rights management, user data stores, and metadata stores.
- Software as a service (SaaS)—including content metadata stores, metadata caches, and relationship graph data stores.
Amazon DocumentDB is a fully managed document database service. It is scalable, highly-available, and compatible with MongoDB. With it, you can store, index, and query JSON files. With DocumentDB, you can scale your compute and storage resources separately for maximum flexibility.
Use cases of Amazon DocumentDB include:
- Content and catalog management—including online publications, point-of-sale terminals, and digital archives.
- Profile management—including user preferences, authentication profiles, and online transactions.
- Mobile and web applications—including applications that demand high-performance and low-latency with millions of requests per second.
Amazon ElastiCache is a fully managed, in-memory data store service. It is compatible with both Redis and Memcached. ElastiCache automates setup, hardware provisioning, configuration, monitoring, updates, and backup and recovery processes. With ElastiCache you can scale both write and memory processes through sharding and data replication.
Use cases of Amazon ElastiCache include:
- Session stores—for web applications and sites.
- Gaming—including leaderboards and chats.
- Geospatial services—including real-time mapping and location.
- Real-time analytics—including Internet of things (IoT) sensor processing and AI applications.
Amazon Neptune is a fully managed graph database service. It enables you to create and run applications using highly-connected data sets. It supports the storage of massive relationship data sets with low-latency access. Neptune supports a variety of graph models and languages, including RDF, SPARQL, and Gremlin. It includes features for point-in-time recovery, read replicas, and continuous backup.
Use cases for Amazon Neptune include:
- Social networking—including user profiles and content prioritization.
- Recommendation engines—including storage of customer contacts, purchase histories, and customer preferences.
- Fraud detection—including fraud related to overlapping email addresses, IP addresses, or credit card numbers.
- Knowledge graphs—including product catalogs or wikis.
- Life sciences—including disease models, gene patterning, or research catalogs.
- Network and IT operations—including creating network visibility, monitoring, or forensic analysis.
Amazon Timestream is a fully managed, time-series database service. It enables you to store, process, and analyze up to 1,000X better query performance at 90% lower cost, compared to relational databases offered on AWS. Timestream provides automatic hardware provisioning, updates, setup and configuration, and data tiering.
Use cases for Amazon Timestream include:
- DevOps—supports performance monitoring and management, network optimization, and server monitoring.
- IoT applications—supports IoT analytics for the implementation of smart devices, such as thermostats or motion sensors.
- Application monitoring—supports clickstream monitoring and analysis.
- Industrial telemetry—including monitoring of industrial equipment, fleet management, trade monitoring, or route optimization.
Amazon Quantum Ledger Database (QLDB)
Amazon (QLDB) is a fully managed, serverless ledger database service. You can use it to track application data changes with a verifiable history. With QLDB, you can avoid the need to build custom ledger applications and associated verification tools. You can query data in QLDB using a SQL-like API.
Use cases for Amazon QLDB include:
- Finance—including credit and debit transactions.
- Insurance—including claim transactions and auditing.
- HR and payroll—including employee benefits, performance histories, or certifications.
- Retail and supply chain management—including batch tracking, product recall processes, and shipping details.
AWS Database Migration Service
Once you’ve decided which services are best for your needs, you need to decide what data you want to migrate and how. Depending on the databases you are currently using, this can be as simple as creating a backup and importing that backup to the new database service. Or, it could require reformatting or re-scheming data.
To make migration easier, Amazon offers the AWS Database Migration Service. This service is free for the first six months of many AWS database services, including Aurora, DynamoDB, and Redshift. With this service, you can generally keep your databases operational during migration. This means minimal downtime and enables you to reduce revenue loss and disruptions to productivity.
Database Migration Service supports most commercial and open-source databases for easy transfer. This includes both homogeneous migrations, for example, SQL Server to SQL Server, or mixed migrations, for example, Oracle to Amazon Aurora. You can also use it to continuously replicate and consolidate databases to a data warehouse via Amazon S3 and RedShift.
AWS Database Migration Service Benefits
Using AWS Database Migration Service can provide several distinct benefits. The most notable benefits include:
Simple to use
Using this service doesn’t require the installation of applications or drivers. It also generally doesn’t require you to make changes to your existing databases. The service is easily launched and managed from the AWS Management Console and once started is fully managed.
To get started from the console, you only need to define your migration parameters in a new migration task. This involves setting up connections between databases and choosing replication instances. Then, you can use these parameters as a template to perform migration tests or multiple live migrations using the same settings.
Any changes you make during the migration process are automatically, and continuously replicated to your target service. This enables you to continue using existing databases up until you are ready to go live with your replacement services. You also have the option of leaving the sync operational indefinitely. This is particularly useful for mission-critical applications that you wish to create a failover for.
AWS Database Migration Service is free for the first six months (depending on the service you’re migrating to). You are only responsible for the compute resources and log storage used during migration. All inbound data transfers are free. Even compute and log storage costs are very low, however. For example, a terabyte-sized migration can generally be done for around USD $3.
For a custom analysis of what migration with this tool might cost, you can check out Amazon’s TCO calculator.
Supports widely used databases
AWS Database Migration Service can migrate data between most commonly used databases. You can use it to easily move databases between platforms or to migrate data between services. For example, you can move data from EC2 to Amazon RDS or from RDS to EC2. You can also use it to transfer data between database types, including NoSQL, SQL, and text-based stores.
When you use the Database Migration Service, your migration is performed in a highly resilient and self–healing way. The service achieves this by continually monitoring your source and target databases, network connectivity, and replication instances. This ensures everything runs smoothly.
However, if issues do arise, the service automatically detects the problem and takes action to correct it or notify you. Typically, this means automatically restarting the migration process from where the issue occurred.
AWS Database Management with Cloud Volumes ONTAP
NetApp Cloud Volumes ONTAP, the leading enterprise-grade storage management solution, delivers secure, proven storage management services on AWS, Azure and Google Cloud. Cloud Volumes ONTAP supports up to a capacity of 368TB, and supports various use cases such as file services, databases, DevOps or any other enterprise workload, with a strong set of features including high availability, data protection, storage efficiencies, Kubernetes integration, and more.
In particular, Cloud Volumes ONTAP helps in addressing database workloads challenges in the cloud, and filling the gap between your cloud-based database capabilities and the public cloud resources it runs on.
Want to get started? Try out Cloud Volumes ONTAP today with a 30-day free trial
Learn More About AWS Databases
In this article, we offered an overview of eight AWS databases, including features, capabilities, and use cases. We also provided a brief guide about AWS migration services, and how you can leverage Cloud Volumes ONTAP to address the challenges of managing databases in the AWS cloud.
There’s a lot more to learn about AWS databases. To continue your research, take a look at the rest of our blogs on this topic:
AWS MySQL: MySQL as a Service vs. Self Managed in the Cloud
You can set up MySQL on AWS on your own, or you can use MySQL databases as a service. The latter is offered as a service called AWS RDS, which enables you to run a managed database instance. Another option is to use Amazon Aurora, an elastic database service which is fully compatible with MySQL.
If you’re looking for information about RDS and Aurora for MySQL, check out our article: AWS MySQL: MySQL as a Service vs. Self Managed in the Cloud.
Overcome Amazon RDS Instance Size Limits with Data Tiering to Amazon S3
Enterprise-grade databases pose a number of challenges, including system performance, security and compliance, and data retention. To do this, you need to understand storage requirements and the key considerations to optimizing costs.
This article explains why Amazon RDS size limits are restrictive, and make the case for hosting your databases directly on EC2. We also show how Cloud Volumes ONTAP can help you avoid reaching RDS limits while controlling costs.
DynamoDB Pricing: How to Optimize Usage and Reduce Costs
DynamoDB is a NoSQL database platform offered by AWS. DynamoDB is suitable for mobile backends, serverless web apps, microservices, and operations that need low-latency data access. However, DynamoDB pricing can quickly get out of hand when you are dealing with demanding workloads and unpredictable peaks.
This post explains how the DynamoDB pricing model works, and how to optimize DynamoDB storage utilization.
SQL Server in AWS: Two Deployment Options
AWS offers two main options for SQL deployments. To save time on management tasks, you can deploy a managed database using Amazon RDS. Alternatively, you can leverage more control using EC2 and EBS to deploy a self-managed SQL Server in AWS.
Read more: SQL Server in AWS: Two Deployment Options
AWS Oracle RDS: Running Your First Oracle Database on Amazon
Amazon Relational Database Service (RDS) is a managed service for relational databases such as Oracle. You can use the AWS platform to backup, monitor, secure, configure, and scale your Oracle workloads.
This post reviews the features and licenses offered for relational Oracle databases on AWS RDS, and walks you through the process of creating an RDS DB instance.
AWS Database as a Service: 8 Ways to Manage Databases in AWS
With AWS Database as a Service (DBaaS), cloud users can use cloud-based databases without having to worry about the underlying infrastructure. To make the best use of these services, it’s beneficial to get familiar with the functionalities, capabilities, use cases, and management requirements of each particular DBaaS.
This article explores eight types of AWS Database as a Service such as key-value, in-memory and relational databases, and details case studies of their deployments. In addition, it shows how Cloud Volumes ONTAP builds on and enhances the native AWS DBaaS storage capabilities.
AWS NoSQL: Choosing the Best Option for You
NoSQL databases support large volumes of data in a way that is flexible and highly performant. With these databases, organizations have been able to capitalize on data and big data in a way they could not with traditional, relational databases.
This article explains what NoSQL databases are, what types of NoSQL databases are available in AWS, and what services you can use to deploy a NoSQL database in AWS.
Read more: AWS NoSQL: Choosing the Best Option for You
AWS Big Data: 6 Options You Should Consider
Big data is a valuable tool for any modern business provided they can store, process, and analyze it efficiently. For many organizations, these tasks are impossible without the scalability and support of cloud services like those offered by AWS.
This article explains what kind of support AWS offers for big data processes and introduces some of the services commonly used for big data analytics.
Read more in AWS Big Data: 6 Options You Should Consider
Amazon DocumentDB: Basics and Best Practices
Amazon DocumentDB is a fully managed service that allows cloud users to access, migrate, and host the workloads of MongoDB, a popular NoSQL database engine.
This article dives into DocumentDB key features and architecture and how it differs from DynamoDB. It also details best practices to successfully deploy MongoDB on AWS, and looks at how Cloud Volumes ONTAP enhances DocumentDB performance and helps overcome major database workload challenges in the cloud.
Read more: Amazon DocumentDB: Basics and Best Practices
Database Case Studies with Cloud Volumes ONTAP
Since most major enterprises use databases, there are many database case studies that can be turned to for insight into how to manage database storage in cloud computing.
In this blog we look at the database case studies of six major enterprises that are using Cloud Volumes ONTAP as the management layer for their cloud-based database storage. See how they leverage this data management platform to decrease database costs, seamlessly migrate from the data center to the cloud, increase hybrid manageability, enhance the dev/test process, and more.
Read more in Database Case Studies with Cloud Volumes ONTAP.
AWS Data Analytics: Choosing the Best Option for You
Big data solutions help organizations to efficiently store, catalogue, search, and analyze their data. AWS offers a wide range of services, each offering different capabilities. This article introduces common AWS Data Analytics offerings, and provides assessment questions.
AWS Oracle: Two Options for Migrating Your Oracle DB to Amazon
Oracle databases are no longer restricted to on-premises operations or even the Oracle cloud. You can deploy Oracle on various cloud platforms, including Amazon Web Services (AWS). There are two main methods to run Oracle on AWS. The first leverages Amazon EC2 resources, and the second is offered by Relational Database Service (RDS) as a managed service.
AWS ElastiCache for Redis: How to Use the AWS Redis Service
AWS ElastiCache for Redis is the fully managed service for Redis, the open-source database and cache technology fast growing in importance in enterprise DevOps deployments. Using AWS ElastiCache for Redis, engineers can easily manage all aspects of their Redis clusters being deployed on AWS, reducing operational costs for key tasks such as monitoring, maintenance, backing up data, recovering from failures, and updating software.
In this blog we take a closer look at Redis, AWS ElastiCache for Redis, and how they can be used as critical parts of AWS database deployment, with a full step-by-step walkthrough to help you get started.
MySQL Database Migration: Amazon EC2-Hosted vs. Amazon RDS
Cloud service providers offer managed database services that enable customers to easily deploy a cloud database without provisioning and configuring physical hardware and infrastructure. On AWS, that option is Amazon Relational Database service (Amazon RDS). But there is another option: deploying a database hosted on native Amazon EC2 instances.
What are the tradeoffs between these two deployment models? In this post we give a side-by-side comparison of the costs, availability, and backup capabilities of these two options. Plus, find out how Cloud Volumes ONTAP makes deploying your MySQL database migration to AWS easier than ever and less expensive.
AWS PostgreSQL: Managed or Self-Managed?
Amazon Web Services (AWS) provides PostgreSQL as a fully managed AWS database service, using Amazon Relational Database Service (RDS) and Amazon Aurora, an Amazon-developed database engine which is fully compatible with PostgreSQL.
Learn how AWS provides managed PostgreSQL with RDS and Aurora, and discover a second option—running PostgreSQL on an Amazon EC2 compute instance with EBS storage, to achieve the same level of control as in your local data center.
Read more: AWS PostgreSQL: Managed or Self-Managed?
See Our Additional Guides on Key Cloud Storage Topics
We have authored in-depth guides on several other topics that can also be useful as you explore the world of cloud storage.
File shares support some of the most important workloads that enterprise businesses rely on, and the resources of the public cloud have created interesting new possibilities. Every major public cloud provider now offers its own cloud file sharing service, each with its own target workloads and considerations. But not every enterprise will find what they’re looking for in a fully managed, all-cloud service.
See top articles in our cloud file sharing guide:
- File Share Service Challenges in the Cloud
- Cloud File Sharing Services: Open-Source Solutions
- Cloud Availability Nightmares and How to Avoid Them in Cloud File Sharing
Multicloud strategies are becoming more popular as organizations seek to optimize their cloud services and deployments. These strategies can help you prevent vendor lock-in, increase your flexibility, and help you optimize costs.
This guide explains what multicloud storage is, how it works, what it’s used for, the core requirements for this storage, and how Cloud Volumes ONTAP supports it.
See top articles in our multicloud storage guide:
- One Cloud Out of Many: Why Enterprises Are Turning to Multicloud and Hybrid Cloud Architectures
- Multicloud Architecture: Partitioned, Cloud Burst and DR
- Multicloud Deployment: Creating a Plan With Cloud Volumes ONTAP
Snapshots are a common method for natively backing up cloud data and services. This method enables you to save point in time backups which can be restored when needed.
This guide explains what types of storage snapshots are available, what AWS snapshots are, and how to use AWS snapshots.
See top articles in our AWS snapshots guide:
- Azure and AWS Snapshots Deep Dive: Cloud Volumes Snapshots
- Snapshots Deep Dive: AWS Snapshots and Azure Snapshots
- Understanding AWS Snapshot Pricing: Data Transfer and Storage Costs
Azure provides a wide variety of services to its users to help you manage your cloud data and services reliably. Azure Backup is one such service that can help provide data loss protection and peace of mind.
This guide explains what Azure Backup is and how to use it to backup your Azure data.
See top articles in our Azure Backup guide:
- Storage Options for Lower Azure Storage Costs and Azure Backup Costs
- The 5 Enterprise-Grade Azure Features You Need to Know About: Azure Backup, Security, and More
- Using Azure Backup Server to Backup Workloads and Files to Azure
Azure File Storage
Storing file data in Azure is simple through Azure File Storage service. This service enables you to store files across cloud and on-premises resources, enabling you to flexibly and securely share data and workflows.
This guide explains what Azure File Storage is, common use cases for Files, management concepts and components of the service, how data is accessed and the architecture of the service, and some best practices for securing your data.
See top articles in our Azure file storage guide:
Azure Files is one of several storage services available to users in Azure. It is a service designed to replicate file shares like those commonly used on premises. With this service, you can smoothly transition your files to the cloud and allow file sharing across your teams.
This guide explains what Azure Files is, how it complements other storage services, pricing and use cases for Files, and pros and cons you should be aware of.
See top articles in our Azure Files guide:
- Azure NetApp Files Register
- SMB File Sharing
- NFS and SMB - A Simple File Service Environment in Azure.
Nearly every production cloud deployment has one or more databases. These tools provide support for applications, enable workloads, and organize your data meaningfully. Having databases available that support all your needs is essential and Azure offers a range to choose from.
This guide explains what Azure database workloads are supported, how databases work in Azure, and what services are available.
See top articles in our Azure database guide:
- Azure Oracle: Your First Oracle Database on Azure
- Azure Database Migration Service: The Ultimate Guide
- Azure SQL Database: 18 Options for SQL Server on the Cloud
Google Cloud offers a variety of storage options for you to choose from. These services form the base of many other services in the cloud and understanding what your options are can help you manage your cloud more efficiently.
This guide explains what Google Cloud Storage options exist and their common uses.
See top articles in our Google Cloud storage guide:
- Cloud File Sharing Services: Google Cloud Filestore
- Google Cloud Storage Encryption: Key Management in Google Cloud
- Google Cloud Storage Pricing: Get the Best Bang for Your Buckets
Google Cloud’s specialty is flexibility and integration of services and this extends to its database services. In Google Cloud you have a wide variety of database deployments, models, and support to choose from.
This guide explains your options for deploying databases in the cloud, what Google Cloud database services are available, and how to choose the right service for you.
See top articles in our Google Cloud database guide:
- Google Cloud SQL: MySQL, Postgres and MS SQL on Google Cloud
- SQL Server on Google Cloud: Managed Service Vs. Managed Storage
- Google Cloud SQL Pricing, Quotas, and Limits: A Cheatsheet for Cost Optimization
Software developers and DevOps engineers are packaging applications into lightweight units called containers. Kubernetes helps manage and scale containers across clusters of physical machines.
In this environment, Kubernetes storage becomes a significant challenge. By default, containers are ephemeral, meaning that any transient data on the container is lost when it shuts down. However, Kubernetes provides several options for persistent storage.
See top articles in our Kubernetes guide: