Design a Data Pipeline for Time Series Data (Temperature Sensor Data)
Scenario to design the System:
------------------------------------------------------
The requirement is to collect and store temperature readings from potentially thousands of sensors.
We also need to be able to quickly retrieve a reading for a given sensor and timestamp.
We need to keep these for 30 days, after which they can be deleted.
Architecture:
----------------------------
https://github.com/SatadruMukherjee/D...
You can refer these videos for practical implementation of different components of the architecture discussed in this video--
Change Data Capture for DynamoDB Streams using Amazon Kinesis in-depth intuition
• Change Data Capture for DynamoDB Stre...
What is DynamoDB TTL (Time To Live?) | Overview and Demo | Part 1
• What is DynamoDB TTL (Time To Live?) ...
What is DynamoDB TTL (Time To Live?) | Overview and Demo | Part 2
• What is DynamoDB TTL (Time To Live?) ...
Learn System Design with AWS:
-------------------------------------------------------------
• Data Architect 101 for Data Engineers