Secure HTAP data architecture using TiDB Cloud
Publication date: June 13, 2023 (Diagram history)
This reference architecture outlines a modern hybrid transactional/analytical processing (HTAP) data stack on AWS. It uses TiDB, an advanced, open-source, distributed SQL database that powers REST and GraphQL APIs, and near real-time analytics.
Secure HTAP data architecture using TiDB Cloud diagram
The following steps describe the architecture:
-
Create RESTful microservices using Amazon API Gateway or AWS AppSync and AWS Lambda. It can use TiDB Cloud as the online transactional processing (OLTP) database. Lambda receives all requests and sends them to the TiDB Server using an Amazon Virtual Private Cloud (Amazon VPC) endpoint.
-
Powered by AWS PrivateLink, the endpoint connection is secure and private and does not expose your data to the public internet. The AWS PrivateLink connection also supports a secure connection between VPCs with overlapping Classless Inter-Domain Routing (CIDR).
-
Equipped with a massively parallel processing (MPP) engine, TiDB can efficiently handle both OLTP and online analytical processing (OLAP) workloads. TiDB uses row-based storage (TiKV) for OLTP workloads and column-based storage (TiFlash) for OLAP workloads. Data is asynchronously replicated to TiFlash using an extended Raft consensus algorithm for strong consistency. TiFlash is a separate set of nodes that isolates workloads and performance impact on the OLTP system.
-
Build business intelligence (BI) reports using AWS Quick. Queries on TiDB are internally routed to TiFlash and optimized to handle analytical workloads. TiDB provides unified architecture and zero-ETL integration between OLTP and OLAP workloads.
Further reading
For additional information, see the following resources: