Advancing Spark - Rethinking ETL with Databricks Autoloader
Figuring out what data to load can be tricky. but Databricks have the answer! The Databricks Autoloader function in Azure uses Event Grid too automatically ingest files as they land - rather than building out file-watching, polling functionality manually.
Join Simon as he runs through the setup of autoloader and looks at some different ways of using the tool.
For more information about Autoloader - see: https://docs.microsoft.com/en-us/azur...
Otherwise, don't forget to subscribe for more Sparky goodness, or check out our blogs at: https://www.advancinganalytics.co.uk/...