Amazon Kinesis Data Firehose: 7 Things You GOT To Know 🔥 | AWS
Amazon Kinesis Data Firehose allows you to load real-time streams into data lakes and analytics services.
Hi Guys, this is Abi from GokceDB and in this video, you are going to learn 7 things about Amazon Kinesis Data Firehose in AWS. Let's get into it.
1. Kinesis Data Firehose is a fully managed service that makes it easy to capture, transform, and load large volumes of streaming data from hundreds of thousands of sources.
2. With just a few clicks in the AWS Management Console, you can create a delivery stream to load data into 30+ destinations such as Amazon S3, Redshift, MongoDB, and more.
3. You can specify a batch size or batch interval to control how quickly data is uploaded to destinations. For example, if you set the batch interval to 60 seconds, you'll receive new data within 60 seconds of sending it to your delivery stream.
4. The service supports common compression algorithms such as Zip, GZip, and Snappy.
5. Your delivery streams automatically scale up and down to handle gigs per second of input data rate and maintain data latency at your specified levels.
6. Data Firehose can convert the format of incoming data from JSON to Apache Parquet or Apache ORC formats before storing the data in Amazon S3. This can help you save on storage and analytics costs.
7. Amazon Kinesis Data Firehose exposes several metrics through the console and Amazon CloudWatch, including the volume of data submitted, the volume of data uploaded to the destination, time from source to destination, and more.
In summary, Amazon Kinesis Data Firehose is a streaming ETL solution that allows you to create a delivery stream, specify a destination, and start streaming real-time data with just a few clicks.