Generic Job to load the data from s3 to Snowflake using copy command & capture the audit logs
In this video , I have discussed about a framework which can be used to load data of diff format from a s3 location to a snowflake table.
Prerequisite:
----------------------
Saving rejected rows while executing copy command | Snowflake
• Saving rejected rows while executing ...
Key Pair Authentication in Snowflake for Linux machine from scratch
• Key Pair Authentication in Snowflake ...
Architecture:
----------------------
https://github.com/SatadruMukherjee/D...
Snowflake Code used:
----------------------------------------
https://github.com/SatadruMukherjee/D...
Python Code used:
--------------------------------
https://github.com/SatadruMukherjee/D...
Below columns should be present in the config file:
------------------------------------------------------------------------------------------
DATABASE
SCHEMA
TABLE_NAME
STAGE_OBJECT
S3_FILE_PATH_TO_BE_APPENDED_WITH_STAGE_OBJECT
WAREHOUSE
SNOWFLAKE_ROLE
FILE_FORMAT
PATTERN
Check this playlist for more Data Engineering related videos:
• Demystifying Data Engineering with Cl...
Snowflake Complete Course from scratch with End-to-End Project with in-depth explanation--
https://doc.clickup.com/37466271/d/h/...
🙏🙏🙏🙏🙏🙏🙏🙏
YOU JUST NEED TO DO
3 THINGS to support my channel
LIKE
SHARE
&
SUBSCRIBE
TO MY YOUTUBE CHANNEL