Data is an integral part of any application and the performance of the application depends a lot upon the data store used to save it for processing. Initially, a lot of apps were using relational databases but the needs of the applications are now changing. A well-architected application with variation in the data model is the need of the hour. One size fits all database no longer works in this era.
This does not mean relational database does not fit in the current era. They are still in use and still power a lot of apps. Like any other aspect of our applications, the database is also designed to solve a particular problem and are built for a purpose. In my opinion, it is very important to understand and match the purpose with the use case and the problem you are trying to solve. This will help you to write scalable and high performant applications.
Welcome to the Era of Purpose Built Database
In this new world, everyone wants to build a huge scale internet application. These application requires a diverse set of data models and have a diverse set of tech stack requirement too. We have started using micro-services, micro-frontends to solve the diverse tech stack requirement but what about data. We are still using the same old generic database solution and trying to make it fit for the purpose it is not designed for. The diverse set of data models is the need of the hour to built those high performant applications we are working on. There are many types of databases available today and each of these solves a purpose. Following is the list:-
- Relational: Enables you to define database schema as well as relational constraints on the schema.
- Key-Value: Enable you to have a fast read/write. They are highly partitioned and can scale quickly.
- Graph: Relations are the first-class citizen in graph databases. You can have properties on relations and can do traversal queries over relation. Data is modelled and navigated as a graph.
- Document: De-normalised data stored in JSON documents. This model maps to the tendency of humans to model the data in their mind. They have flexible schema and expressive query.
- In-Memory: This type is good for low latency and high scale operations per second. Good for storing data for something like cache.
- Time-Series: Provides performing storage for the data that accumulates over time. Time is the primary axis and this type is good for performing time-based data modelling.
- Ledger: Immutable and cryptographically verifiable data store. It only allows you to append the data in the store and keeps the ledger as the name suggest for all the actions that have taken place in the past
- Search: Many applications output a lot of logs or need functionality to perform a search in tons of data. These data stores come up with advanced search capabilities to provide support in this scenario.
We have talked about the various database types based on the purpose. Following is the list of the offerings by AWS for purpose-built databases:-
| Database type | AWS service | Use Case |
|---|---|---|
| Relational | Amazon Aurora, Amazon RDS, Amazon Redshift | Traditional applications, ERP, CRM where referential integrity is important |
| Key-Value | Amazon DynamoDB | High traffic web apps |
| Graph | Amazon Neptune | Fraud detection, social network, knowledge graphs, recommendation engine |
| Document | Amazon DocumentDB (with MongoDB compatibility) | content management, user profiles |
| In-Memory | Amazon ElastiCache for Memcached and Redis | caching, session management |
| Time-Series | Amazon Timestream | IoT applications, time-series data like stock tickers |
| Ledger | Amazon QLDB | record systems, financial transactions etc. |
| Search | Amazon Elasticsearch | full-text search and advanced analytics |
AWS provides the above-mentioned options to build modern applications and support various types of data models required in building those applications. Like we are not building monolithic applications anymore and we are migrating towards micro-services, we should also start looking into the need for purpose-built databases.