More about AWS database
- Amazon DocumentDB: Basics and Best Practices
- AWS NoSQL: Choosing the Best Option for You
- AWS Big Data: 6 Options You Should Consider
- Oracle on AWS: Managed Service vs. Managed Storage Options
- AWS Oracle RDS: Running Your First Oracle Database on Amazon
- SQL Server in AWS: Managed Service Vs. Managed Storage
- DynamoDB Pricing: How to Optimize Usage and Reduce Costs
- AWS MySQL: Two Ways to Enjoy MySQL as a Service
- AWS Oracle: How to Lift and Shift Your Oracle DB to Amazon
- How to Overcome Amazon RDS Instance Size Limits
Of Amazon’s 212 cloud computing services, fifteen are purpose-built database engines. In this post, we focus on eight AWS database services: Amazon RDS, Amazon Aurora, Amazon DynamoDB, Amazon DocumentDB, Amazon ElastiCache, Amazon Neptune, Amazon Timestream, and Amazon Quantum Ledger Database (QLDB).
We offer a review of the functions, feature, capabilities and use cases of each of these services, to help you choose the database that is a perfect match for your use case and project. In addition, we’ll show how NetApp Cloud Volumes ONTAP can help solve AWS database migration and management challenges.
In this article, you will learn:
- Types of databases on AWS
- AWS database services
- AWS migration services
- AWS database management with Cloud Volumes ONTAP
Types of Database Services Offered on AWS
AWS offers a wide range of database services for you to choose from. The service fall into two groups: relational and non-relational (NoSQL).
In this section we provide a breakdown of Amazon database services into relational and non-relational, in the AWS Database Services section below you can learn about selected services in more detail.
AWS Relational Database Services
Relational databases store data in tabular form with columns and rows, and can be queried using the SQL query language. In these databases, columns represent attributes and rows represent records. Each field in the table is a data value.
Use cases for relational databases on Amazon include:
- Enterprise resource planning (ERP) apps
- Customer relationship management (CRM) apps
- Finance data
- Data warehousing
The primary Amazon services providing relational databases are:
- Amazon Aurora
- Amazon RDS
- Amazon Redshift
AWS NoSQL Database Services
Relational databases are not suitable for many use cases, especially those requiring very high performance or dynamic scalability. NoSQL, or non-relational databases, break the paradigm of storing data in tables with columns and rows, allowing them to distribute and process data more efficiently. NoSQL is commonly used to handle big data - large volumes of unstructured or semi-structured data.
The table below shows the main NoSQL databases services offered by AWS.
Type of Database
Key-value databases store data as a collection of key-value pairs with the key as an ID. These databases can store various types of data, including simple and compound objects.
● Real-time bidding
● eCommerce shopping carts
● Product catalogs
● Customer preferences
● Amazon DynamoDB
Document databases store data in JSON or JSON-like documents. You can query data using the same document-model format used in programming applications.
● Content management systems
● Customer profiles and personalization
● Mobile apps
● Amazon DocumentDB
In-memory databases store data in-memory for low-latency access. You can use these stores as a database, cache, message broker, or queue.
● Session stores
● Geospatial services
● Pub/sub messaging
● Real-time streaming
● Amazon ElastiCache for Memcached
● Amazon ElastiCache for Redis
Graph databases are a type of NoSQL (non-relational) database. This database type represents relationships directly. You can query data with specific graph languages.
● Fraud detection
● Social networking
● Recommendation engines
● Knowledge graphs
● Data lineage
● Amazon Neptune
Time-series databases store data in time-order and as append-only. You can query data over various time intervals.
● Application monitoring
● Industrial telemetry
● IoT applications
● Amazon Timestream
Ledger databases store data in an immutable, transparent, and cryptographically verifiable log. This log is owned by a trusted central authority to ensure provenance.
● Insurance claims
● HR and payroll
● Retail inventories
●Amazon Quantum Ledger Database
Note: While we listed in-memory databases under the non-relational category, there are in-memory databases that are relational, SQL databases. A notable example is SAP HANA. You can run SAP HANA on AWS using the managed SAP HANA instance.
AWS Databases Services
Once you understand what options are available to you for databases in AWS, you can begin narrowing them down. The following services are some of the most commonly used. Keep in mind when reviewing these descriptions that frequently, AWS customers implement multiple database types to meet their needs. You too should consider multiple options if one doesn’t meet all of your needs.
Amazon RDS is a managed, relational database service that includes six different database options. These include AWS Oracle, PostgreSQL, AWS MySQL, MariaDB, SQL Server, and Amazon Aurora. You can manage these database engines from a centralized management console, a command-line interface, or via API calls. When using this service, many administrative tasks are automated, including database setup, hardware provisioning, backup, and updating.
Use cases of Amazon RDS include:
- Web and mobile applications—provides the scalability, availability, and throughput needed for enterprise-grade applications.
- eCommerce applications—provides flexibility, security, and PCI compliance needed for eCommerce.
- Mobile and online games—provides high-throughput and availability to ensure that games remain online and responsive to players.
Amazon Aurora is a fully managed relational database engine designed specifically for AWS. It is MySQL and PostgreSQL compatible with minor changes to your source database. Aurora includes features for self-healing, fault tolerance, point-in-time recovery, and continuous backup.
Use cases for Amazon Aurora include:
- Enterprise applications—including customer relationship management and enterprise resource planning solutions.
- Software as a Service (SaaS) offerings—including those requiring significant storage and compute scalability.
- Web and mobile gaming applications—including those requiring massive storage, high throughput, and high-availability.
Amazon DynamoDB is a fully managed, document and key-value database. It includes features for multi-master, multi-region used along with built-in security, automated backup and restoration, and in-memory caching. DynamoDB can provide support for serverless web apps, microservices, and mobile backends.
Use cases of Amazon DynamoDB include:
- Ad tech—including clickstreams, user events, and user profiles.
- Gaming—including leaderboards, player data stores, and game states.
- Retail—including online shopping carts, inventory tracking, and customer profiles.
- Banking and finance—including event-driven transaction processing, fraud detection, and change data capture.
- Media and entertainment—including digital rights management, user data stores, and metadata stores.
- Software as a service (SaaS)—including content metadata stores, metadata caches, and relationship graph data stores.
Amazon DocumentDB is a fully managed document database service. It is scalable, highly-available, and compatible with MongoDB. With it, you can store, index, and query JSON files. With DocumentDB, you can scale your compute and storage resources separately for maximum flexibility.
Use cases of Amazon DocumentDB include:
- Content and catalog management—including online publications, point-of-sale terminals, and digital archives.
- Profile management—including user preferences, authentication profiles, and online transactions.
- Mobile and web applications—including applications that demand high-performance and low-latency with millions of requests per second.
Amazon ElastiCache is a fully managed, in-memory data store service. It is compatible with both Redis and Memcached. ElastiCache automates setup, hardware provisioning, configuration, monitoring, updates, and backup and recovery processes. With ElastiCache you can scale both write and memory processes through sharding and data replication.
Use cases of Amazon ElastiCache include:
- Session stores—for web applications and sites.
- Gaming—including leaderboards and chats.
- Geospatial services—including real-time mapping and location.
- Real-time analytics—including Internet of things (IoT) sensor processing and AI applications.
Amazon Neptune is a fully managed graph database service. It enables you to create and run applications using highly-connected data sets. It supports the storage of massive relationship data sets with low-latency access. Neptune supports a variety of graph models and languages, including RDF, SPARQL, and Gremlin. It includes features for point-in-time recovery, read replicas, and continuous backup.
Use cases for Amazon Neptune include:
- Social networking—including user profiles and content prioritization.
- Recommendation engines—including storage of customer contacts, purchase histories, and customer preferences.
- Fraud detection—including fraud related to overlapping email addresses, IP addresses, or credit card numbers.
- Knowledge graphs—including product catalogs or wikis.
- Life sciences—including disease models, gene patterning, or research catalogs.
- Network and IT operations—including creating network visibility, monitoring, or forensic analysis.
Amazon Timestream is a fully managed, time-series database service. It enables you to store, process, and analyze up to 1,000X better query performance at 90% lower cost, compared to relational databases offered on AWS. Timestream provides automatic hardware provisioning, updates, setup and configuration, and data tiering.
Use cases for Amazon Timestream include:
- DevOps—supports performance monitoring and management, network optimization, and server monitoring.
- IoT applications—supports IoT analytics for the implementation of smart devices, such as thermostats or motion sensors.
- Application monitoring—supports clickstream monitoring and analysis.
- Industrial telemetry—including monitoring of industrial equipment, fleet management, trade monitoring, or route optimization.
Amazon Quantum Ledger Database (QLDB)
Amazon (QLDB) is a fully managed, serverless ledger database service. You can use it to track application data changes with a verifiable history. With QLDB, you can avoid the need to build custom ledger applications and associated verification tools. You can query data in QLDB using a SQL-like API.
Use cases for Amazon QLDB include:
- Finance—including credit and debit transactions.
- Insurance—including claim transactions and auditing.
- HR and payroll—including employee benefits, performance histories, or certifications.
- Retail and supply chain management—including batch tracking, product recall processes, and shipping details.
AWS Database Migration Service
Once you’ve decided which services are best for your needs, you need to decide what data you want to migrate and how. Depending on the databases you are currently using, this can be as simple as creating a backup and importing that backup to the new database service. Or, it could require reformatting or re-scheming data.
To make migration easier, Amazon offers the AWS Database Migration Service. This service is free for the first six months of many AWS database services, including Aurora, DynamoDB, and Redshift. With this service, you can generally keep your databases operational during migration. This means minimal downtime and enables you to reduce revenue loss and disruptions to productivity.
Database Migration Service supports most commercial and open-source databases for easy transfer. This includes both homogeneous migrations, for example, SQL Server to SQL Server, or mixed migrations, for example, Oracle to Amazon Aurora. You can also use it to continuously replicate and consolidate databases to a data warehouse via Amazon S3 and RedShift.
AWS Database Migration Service Benefits
Using AWS Database Migration Service can provide several distinct benefits. The most notable benefits include:
Simple to use
Using this service doesn’t require the installation of applications or drivers. It also generally doesn’t require you to make changes to your existing databases. The service is easily launched and managed from the AWS Management Console and once started is fully managed.
To get started from the console, you only need to define your migration parameters in a new migration task. This involves setting up connections between databases and choosing replication instances. Then, you can use these parameters as a template to perform migration tests or multiple live migrations using the same settings.
Any changes you make during the migration process are automatically, and continuously replicated to your target service. This enables you to continue using existing databases up until you are ready to go live with your replacement services. You also have the option of leaving the sync operational indefinitely. This is particularly useful for mission-critical applications that you wish to create a failover for.
AWS Database Migration Service is free for the first six months (depending on the service you’re migrating to). You are only responsible for the compute resources and log storage used during migration. All inbound data transfers are free. Even compute and log storage costs are very low, however. For example, a terabyte-sized migration can generally be done for around USD $3.
For a custom analysis of what migration with this tool might cost, you can check out Amazon’s TCO calculator.
Supports widely used databases
AWS Database Migration Service can migrate data between most commonly used databases. You can use it to easily move databases between platforms or to migrate data between services. For example, you can move data from EC2 to Amazon RDS or from RDS to EC2. You can also use it to transfer data between database types, including NoSQL, SQL, and text-based stores.
When you use the Database Migration Service, your migration is performed in a highly resilient and self–healing way. The service achieves this by continually monitoring your source and target databases, network connectivity, and replication instances. This ensures everything runs smoothly.
However, if issues do arise, the service automatically detects the problem and takes action to correct it or notify you. Typically, this means automatically restarting the migration process from where the issue occurred.
AWS Database Management with Cloud Volumes ONTAP
NetApp Cloud Volumes ONTAP, the leading enterprise-grade storage management solution, delivers secure, proven storage management services on AWS, Azure and Google Cloud. Cloud Volumes ONTAP supports up to a capacity of 368TB, and supports various use cases such as file services, databases, DevOps or any other enterprise workload.
In particular, Cloud Volumes ONTAP helps in addressing database workloads challenges in the cloud, and filling the gap between your cloud-based database capabilities and the public cloud resources it runs on.
Want to get started? Try out Cloud Volumes ONTAP today with a 30-day free trial.
Learn More About AWS Databases
In this article, we offered an overview of eight AWS databases, including features, capabilities, and use cases. We also provided a brief guide about AWS migration services, and how you can leverage Cloud Volumes ONTAP to address the challenges of managing databases in the AWS cloud.
There’s a lot more to learn about AWS databases. To continue your research, take a look at the rest of our blogs on this topic:
AWS MySQL: Two Ways to Enjoy MySQL as a Service
You can set up MySQL on AWS on your own, or you can use MySQL databases as a service. The latter is offered as a service called AWS RDS, which enables you to run a managed database instance. Another option is to use Amazon Aurora, an elastic database service which is fully compatible with MySQL.
If you’re looking for information about RDS and Aurora for MySQL, check out our article: AWS MySQL: Two Ways to Enjoy MySQL as a Service.
AWS Oracle: How to Lift and Shift Your Oracle DB to Amazon
Many Oracle databases are still located on-premises, despite the difficulties of on-prem data management. If you are looking to migrate your Oracle database to AWS, there are two main AWS database services that can run Oracle databases: The Relational Database Service (RDS) and Amazon EC2.
Our article: AWS Oracle: How to Lift and Shift Your Oracle DB to Amazon, can help walk you through the process of migrating on-prem Oracle workloads to the cloud.
Overcome Amazon RDS Size Limits with Data Tiering to Amazon S3
Enterprise-grade databases pose a number of challenges, including system performance, security and compliance, and data retention. To do this, you need to understand storage requirements and the key considerations to optimizing costs.
In our article, Overcome Amazon RDS Size Limits with Data Tiering to Amazon S3, we explain why Amazon RDS size limits are restrictive, and make the case for hosting your databases directly on EC2. We also show how Cloud Volumes ONTAP can help you avoid reaching RDS limits while controlling costs.
DynamoDB Pricing: How to Optimize Usage and Reduce Costs
DynamoDB is a NoSQL database platform offered by AWS. DynamoDB is suitable for mobile backends, serverless web apps, microservices, and operations that need low-latency data access. However, DynamoDB pricing can quickly get out of hand when you are dealing with demanding workloads and unpredictable peaks.
This post explains how the DynamoDB pricing model works, and how to optimize DynamoDB storage utilization.
SQL Server in AWS: Managed Service Vs. Managed Storage
Using SQL Server is a popular choice among enterprises running mission-critical databases. When using AWS, databases like these can either be run via the cloud provider’s managed database service, Amazon Relational Database Service (Amazon RDS), or through the managed storage option built on native AWS services such as Amazon EC2 and Amazon EBS.
In SQL Server in AWS: Managed Service Vs. Managed Storage, we provide an overview on how SQL Server in AWS works, taking a closer look at both of the deployment models mentioned above. Plus, we’ll see how Cloud Volumes ONTAP can be used as the management layer for managed storage deployments, adding further benefits such as instant snapshots for data protection, data cloning capabilities, cost-cutting technologies, and business continuity ensuring availability.
AWS Oracle RDS: Running Your First Oracle Database on Amazon
Amazon Relational Database Service (RDS) is a managed service for relational databases such as Oracle. You can use the AWS platform to backup, monitor, secure, configure, and scale your Oracle workloads.
This post reviews the features and licenses offered for relational Oracle databases on AWS RDS, and walks you through the process of creating an RDS DB instance.
AWS Database Migration Service: Copy-Paste Your Database to Amazon
AWS Database Migration Service (DMS) is a service for migrating any data type from any data source to the AWS cloud. DMS supports migration to RedShift, Amazon RDS, Aurora, DocumentDB, and DynamoDB.
You do not need to install agents or alter your source database to use the Database Migration Service. Instead, you can control the migration process from the AWS Management Console. Once started, migration is a managed service that is resilient and self-healing.
AWS Database as a Service: 8 Ways to Manage Databases in AWS
AWS offers the largest variety of Database as a Service (DBaaS) offerings of all major cloud vendors. This includes support for deployments of most complexities and for a growing number of database engines. AWS also offers support for flexible resource provisioning, high-availability, built-in security, and replication.
Oracle on AWS: Managed Service vs. Managed Storage Options
Oracle was one of the first major database choices for enterprise deployment, and it’s still a widely used option throughout industry. The cloud has changed many things, but Oracle remains a viable database model, including on AWS.
There are two popular ways to deploy Oracle on AWS: one involves the use of the AWS managed Amazon Relational Database Service (Amazon RDS), while the other is built on the raw AWS cloud infrastructure, namely AWS Amazon EC2 compute instances.
Which is the right model for deploying Oracle on AWS for your organization? Both come with a measure of success, and both present challenging aspects to take into consideration. In this article we look at both Oracle on AWS deployment models, managed service and managed storage, and the advantages that come with deploying with NetApp Cloud Volumes ONTAP.
AWS NoSQL: Choosing the Best Option for You
NoSQL databases support large volumes of data in a way that is flexible and highly performant. With these databases, organizations have been able to capitalize on data and big data in a way they could not with traditional, relational databases.
This article explains what NoSQL databases are, what types of NoSQL databases are available in AWS, and what services you can use to deploy a NoSQL database in AWS.
Read more: AWS NoSQL: Choosing the Best Option for You
AWS Big Data: 6 Options You Should Consider
Big data is a valuable tool for any modern business provided they can store, process, and analyze it efficiently. For many organizations, these tasks are impossible without the scalability and support of cloud services like those offered by AWS.
This article explains what kind of support AWS offers for big data processes and introduces some of the services commonly used for big data analytics.
Read more in AWS Big Data: 6 Options You Should Consider
Amazon DocumentDB: Basics and Best Practices
Document databases are useful, non-relational tools for storing data in JSON, YAML, or other markup language documents. These databases are distributed, scalable systems that enable you to store hierarchical, semi-structured data with ease. To provide users access to document database benefits, AWS offers DocumentDB.
This article explains what DocumentDB is, how it works, how it compares to DynamoDB, and some best practices for implementing it.
Read more: Amazon DocumentDB: Basics and Best Practices