Streaming Data from Snowflake to Kafka - Capture table deltas to kafka topic
In this video, we will be sending data from Snowflake to Kafka. You will learn how to capture table deltas using Snowflake Streams and sent them to kafka using s3 source kafka connector.
#snowflake #kafka #streaming #snowflaketokafka #maxcoteclearning #maxcotec
Read the article: https://maxcotec.com/learning/snowfla...
Get the code here: https://github.com/maxcotec/snowflake...
timeline:
00:00 intro
00:51 video outline
01:58 data flow diagram
04:01 setup kafka cluster
05:16 setup s3 kafka connector
06:19 create s3 bucket and IAM user
07:36 configure s3 kafka connector
08:58 access connector logs
10:01 snowflake storage integration
16:02 Snowflake streams
16:54 Snowflake streams pricing
17:35 Snowflake streams cont.
20:50 Snowflake to kafka testing
24:40 s3 connector limitation
26:03 Conclusion
28:03 Outro