Building AWS Glue Job using PySpark
The workshop URLs
Part1- https://aws-dojo.com/workshoplists/wo...
Part2- https://aws-dojo.com/workshoplists/wo...
AWS Glue Jobs are used to build ETL job which extracts data from sources, transforms the data, and loads it into targets. The job can be built using languages like Python and PySpark. PySpark is the Python API for Spark and it used for big data processing. It can perform data transformation on large scale data in fast and efficient way. This workshop will be covered in two parts.
Part-1: You learn about setting up a data lake, creating development environment for PySpark and finally building a Glue job using PySpark.
Part-2: You learn about PySpark for various types of transformations especially when using AWS Lake Formation and Glue based data lake. These transformations can be used in AWS Glue job.