View a markdown version of this page

Secure HTAP data architecture using TiDB Cloud - Financial Services Reference Architecture Diagrams

Secure HTAP data architecture using TiDB Cloud

Publication date: June 13, 2023 (Diagram history)

This reference architecture outlines a modern hybrid transactional/analytical processing (HTAP) data stack on AWS. It uses TiDB, an advanced, open-source, distributed SQL database that powers REST and GraphQL APIs, and near real-time analytics.

Secure HTAP data architecture using TiDB Cloud diagram

Requests pass through API Gateway and Lambda to TiDB Cloud over AWS PrivateLink for OLTP and OLAP.

The following steps describe the architecture:

  1. Create RESTful microservices using Amazon API Gateway or AWS AppSync and AWS Lambda. It can use TiDB Cloud as the online transactional processing (OLTP) database. Lambda receives all requests and sends them to the TiDB Server using an Amazon Virtual Private Cloud (Amazon VPC) endpoint.

  2. Powered by AWS PrivateLink, the endpoint connection is secure and private and does not expose your data to the public internet. The AWS PrivateLink connection also supports a secure connection between VPCs with overlapping Classless Inter-Domain Routing (CIDR).

  3. Equipped with a massively parallel processing (MPP) engine, TiDB can efficiently handle both OLTP and online analytical processing (OLAP) workloads. TiDB uses row-based storage (TiKV) for OLTP workloads and column-based storage (TiFlash) for OLAP workloads. Data is asynchronously replicated to TiFlash using an extended Raft consensus algorithm for strong consistency. TiFlash is a separate set of nodes that isolates workloads and performance impact on the OLTP system.

  4. Build business intelligence (BI) reports using AWS Quick. Queries on TiDB are internally routed to TiFlash and optimized to handle analytical workloads. TiDB provides unified architecture and zero-ETL integration between OLTP and OLAP workloads.

Further reading

For additional information, see the following resources: