Control ADF Pipeline by Comparing Last Modified Date & Last Load Date | Real Time Case Study
In this Video I covered the real time scenario by performing below steps.
1. Get File Metadata and Load into Log table for audit purpose.
2. Pull File Last Modified date from Audit table and compare with Last Load date in Target table (Product_sales table)
3. In case Last Modified date is not equals to LoadDate then go ahead and Load the data from csv file to target table once load is done copy the file to Archive Location.
4. In case File Modified date and Load date are same just go ahead and copy the file to archive location.
Control ADF Pipeline by Comparing Last Modified Date & Last Load Date | Real Time Case Study