Anjan GCP Data Engineering

Google Cloud Storage blazing fast uploads and downloads with Python client libraries

Data Lineage | Track Big Query table data Lineage

Write Big Query SQL queries with Gemini assistance

Basics of Service Accounts and IAM Roles

Big Query BigLake Tables

Learn code free Data pipelines with Cloud Data Fusion | Packed with detailed Demos and explanations

Serverless Change Data Capture (CDC) , Replication , Data Synchronisation with Cloud Datastream

ETL nested AVRO files into Cloud SQL, GCS using Dataflow Pipeline

Automate GCP Project’s IAM Roles Snapshot process using Cloud Functions, Bigquery, Cloud Scheduler

GCP Professional Data Engineer, re certified ,Exam experience and my suggestions to Aspirants

Selecting GCP ETL/Data processing services | GCP Professional Data Engineer

GCP Professional Data Engineer | Storage and Database selection process, Test your knowledge

GCP Cloud Logging | Export logs to GCS , Big Query , PubSub for further Analysis or Process

Cloud Big Table - Introduction | Console , CBT - Basic Demo with Examples

Cloud SQL Python connector | Access Cloud SQL Instance Programatically

Google Cloud Storage (GCS) Object Lifecycle Management with Examples

Update on code samples used in demos

GCP Cloud Functions for Big query events | Extract data from big query table to GCS Bucket in CSV

Big Query Editor Schedule SQL/Script

Copy GCS Bucket using Storage Transfer Job

GCP Apache Beam stream data processing pipeline | Pub Sub , Dataflow , Big Query

Connecting Cloud SQL from MySQL workbench

GCP Write curated logs to Big Query using PubSub events - Cloud Function combination

Cloud Spanner Introduction demo with examples

Cloud Scheduler Introduction with simple examples

Import Bulk data from GCS into Spanner Table using Dataflow Jobs

The Secret Manager in GCP to store and access sensitive data

Big Query Clustered Tables with Examples

GCP Dataflow | Apache Beam | Side Inputs | Composite PTransforms

Dataproc Workflow Templates | Execute Jobs DAG on Managed Dataproc Cluster

Schedule secured data load from Google sheets to Big query|Cloud Functions,Cloud Scheduler and more

GCP Dataflow Apache Beam | Fixed windows , Sliding Windows , Session windows

Composer - Airflow https operators, Extract data from web API using https operators with examples

Google Cloud Storage Object Versioning and Auto Class

GCP Big Query table snapshots creation automation | Schedule script using service account

Export and Analyze GCP Billing data with BigQuery and Looker Studio

Schedule Dataproc Workflows using Cloud Scheduler

GCP Professional Data Engineer | Selecting suitable storage and database services based on use case

Running and Scheduling Dataflow Batch Pipelines (Dataflow Python Operator) with Composer

GCP - Dataproc ,PySpark - Shell , PySpark | Job Submit through Script

Big Query Federated queries | Connect and execute Cloud SQL queries from Big Query

GCP Composer | Airflow GCS to BigQuery and BigQuery Operators

GCP Dataproc create cluster using CLI | Run PySpark job through GCP console

GCP Cloud Functions for GCS object events | Load data into Big Query tables against GCS events

GCP Dataflow Streaming data processing demo

GCP Cloud SQL Introduction | Create MySQL Instance and connect it from Cloud Shell

Apache Beam Sliding Windows

Big Query Data Transfer | Python API | Copy dataset

GCP Create Dataproc cluster with Jupyter Notebook installed | Run PySpark Jobs from Notebook

GCP Composer DAG code and deployment to DAGs folder | Python and Bash operator examples

GCP Big Query ,Introduction, create datasets and tables using different options

Import and Export data to and from Cloud SQL Instance

GCP Pub Sub | Python Client Libraries | Create Publisher , Subscriber Apps using Compute VM

GCP Create and deploy Cloud Functions | Introduction