Coding from scratch a realtime Azure data and AI pipeline - the Azure side
In this vid (part 1 of 2), I kick off the resources needed to run a realtime data and AI pipeline and process the inputs ready for data scientists to consume the feed.
The video demonstrates:
Using the AZ CLI in Cloud Shell to generate resources
Using the DataBricks CLI in the Cloud to connect to DataBricks and create custom clusters
Set up a realtime Logic App that gets tweets, filters out any retweets, stores the tweets in an Event Hub, grabs any images from the tweets and stores them in Blob storage
Set up an Azure Function that resizes images to fit within Cognitive Services limitations
Create an Event Grid subscription to new blobs that writes to another Event Hub
Follow the how-to at https://github.com/lockedata/realtime...