Looking to get in touch?
Drop me a line at vishal.bulbule@gmail.com, or schedule a meeting using the provided link [ Ссылка ]
Cricket Statistics Data Pipeline in Google Cloud using Airflow,Dataflow,Cloud Function and Looker Studio
Data Retrieval: We fetch data from the Cricbuzz API using Python.
Storing Data in GCS: After fetching the data, we store it in a CSV file in Google Cloud Storage (GCS).
Cloud Function Trigger: Create a Cloud Function that triggers upon file upload to the GCS bucket. The function will execute when a new CSV file is detected and trigger dataflow job.
Cloud Function Execution: Inside the Cloud Function, we will have code that triggers a Dataflow job. Ensure you handle the trigger correctly and pass the required parameters to initiate the Dataflow job.
Dataflow Job: The Dataflow job is triggered by the Cloud Function and loads the data from the CSV file in the GCS bucket into BigQuery. Ensure you have set up the necessary configurations.
Looker Dashboard: BigQuery serves as the data source for your Looker Studio dashboard. Configure Looker to connect to BigQuery and create the dashboard based on the data loaded.
Github Repo for all code used in this project
[ Ссылка ]
============================================
Associate Cloud Engineer -Complete Free Course
[ Ссылка ]
Google Cloud Data Engineer Certification Course
[ Ссылка ]
Google Cloud Platform(GCP) Tutorials
[ Ссылка ]
Generative AI
[ Ссылка ]
Getting Started with Duet AI
[ Ссылка ]
Google Cloud Projects
[ Ссылка ]
Python For GCP
[ Ссылка ]
Terraform Tutorials
[ Ссылка ]-
Linkedin
[ Ссылка ]
Medium Blog
[ Ссылка ]
Github Repository for Source Code
[ Ссылка ]
Email - vishal.bulbule@techtrapture.com
#dataengineeringessentials #dataengineers #dataengineeringproject #airflow #dataflow #cloudcomposer #bigquery #looker #googlecloud #datapipeline
![](https://s2.save4k.org/pic/UXJxcWgxwu0/maxresdefault.jpg)