Working on Ecommerce dataset in BigQuery with DataPrep
In this video, I walk through my project creating an end-to-end data pipeline on Google Cloud Platform for an ecommerce company. Using BigQuery, Cloud Dataprep, and Cloud Composer, I ingest raw session data, transform it into analytics-ready tables, and orchestrate scheduled runs.
Key highlights:
Ingesting batch and streaming session data from BigQuery
Handling data quality checks and preprocessing in Cloud Dataprep
Building reusable data transformation scripts
Scheduling and monitoring pipeline runs with Cloud Composer
Delivering analytics tables to business users on BigQuery
This project gave hands-on experience with GCP big data services to build a scalable data pipeline.
#googlecloud #bigdata #analytics #datapipeline