How to Build Data Pipeline to Perform S3 to S3 ETL using AWS GLUE | Data Engineering Series | Cloud
#dataengineering #data #aws #cloudcomputing #bigdata
Learn how to Transform S3 data using AWS Glue and load the transformed data into S3.
This first introductory demo showcases how to perform a basic transformation on parquet data such as data schema manipulation, filtering and casting data types from an S3 source, and writing the data to the S3 sink.
00:00 - Introduction
00:58 - Create AWS IAM Role for AWS Glue to access S3
02:54 - S3 Data Preview
03:35 - Glue Visual ETL page overview
04:40 - S3 Data load
06:45 - S3 Data Preview
07:35 - Data Transformation using AWS Glue Nodes
13:00 - Data Load into AWS S3
14:00 - Pipeline flow explained
LET'S CONNECT!
📰 LinkedIn ➔ / jayachandra-sekhar-reddy
🐦 Twitter ➔ / reddyjaysekhar
📖Medium ➔ / jay-reddy
📲 Substack➔ https://databracket.substack.com
💁Fiverr ➔ https://www.fiverr.com/jayreddy9
#aws #dataengineering #etl #apachespark #cloudengineering #awsservices #datapipeline