Accessing s3 Data in Spark outside the AWS account using hadoop-aws and aws-java-sdk packages
Have you faced difficulty when you tried to read the s3 data (in AWS Account A) from outside that AWS env. like from local (to do some POC) or Google Colab or from EMR Cluster running in some other AWS Account (Account B)?
If yes , then here is a simple solution for you using which you can read the s3 data in a spark df and write back after processing it from anywhere ..
Code:
https://github.com/SatadruMukherjee/D...
Check this playlist for more AWS Projects in Big Data domain:
• Demystifying Data Engineering with Cl...
🙏🙏🙏🙏🙏🙏🙏🙏
YOU JUST NEED TO DO
3 THINGS to support my channel
LIKE
SHARE
&
SUBSCRIBE
TO MY YOUTUBE CHANNEL