Attend FREE Webinar on Digital Marketing for Career & Business Growth Register Now
Fulfill Your Career Objectives

Take Our Free Counselling Session

  • This field is for validation purposes and should be left unchanged.

AIRflow at Scale: Webinar Recording

AIRflow at Scale: Webinar Recording

This webinar on ‘AIRflow at Scale’ covered the walkthrough of how Qubole extended Apache AIRflow to manage the operational inefficiencies that arise when you manage data pipelines in a multi-tenant environment. And also the process of making the data pipelines robust by adding data quality checks using CheckOperators.

Key Takeaways:

Listed below are the other major key takeaways: 

  • Overview of major type of data pipelines
  • How Qubole manages deployments and upgrades of data pipelines in a multi-tenant environment
  • The Data quality issues which Qubole faces in process of data ingestion or transformation.
  • The approach that Qubole has adopted using Apache Airflow Check operators.
  • Enhancements we had to make to Check operators.
  • Integration of Apache Airflow Check operators with our ETLs.
  • Challenges Qubole faced in developing the alerting framework.
  • The best practices in using Apache Airflow for data quality checks.

Did you find the webinar interesting? Register here for more.

Your Comment

Your email address will not be published.