Utilizing Apache Airflow for Data Ingestion and Data Pipelines

Project Info:

This work consists of frequent usage of Apache Airflow for data ingestion jobs to support ML model pipelines, in addition to data migration tasks from on-prem databases to cloud storage

Project Details:

  • Technologies:Apache Airflow, Google Cloud CLI, Google BigQuery
  • Date:Nov. 2022