AIRflow at Scale: Webinar Recording

by | Jul 13, 2018 | Webinars

< 1 Min Read. |

This webinar on ‘AIRflow at Scale’ covered the walkthrough of how Qubole extended Apache AIRflow to manage the operational inefficiencies that arise when you manage data pipelines in a multi-tenant environment. And also the process of making the data pipelines robust by adding data quality checks using CheckOperators.

Key Takeaways:

Listed below are the other major key takeaways: 

  • Overview of major type of data pipelines
  • How Qubole manages deployments and upgrades of data pipelines in a multi-tenant environment
  • The Data quality issues which Qubole faces in process of data ingestion or transformation.
  • The approach that Qubole has adopted using Apache Airflow Check operators.
  • Enhancements we had to make to Check operators.
  • Integration of Apache Airflow Check operators with our ETLs.
  • Challenges Qubole faced in developing the alerting framework.
  • The best practices in using Apache Airflow for data quality checks.

Did you find the webinar interesting? Register here for more.

Register for FREE Digital Marketing Orientation Class
Date: 30th Jan, 2021 (Sat)
Time: 11 AM to 12:30 PM (IST/GMT +5:30)
  • This field is for validation purposes and should be left unchanged.
We are good people. We don't spam.

You May Also Like…


Submit a Comment

Your email address will not be published. Required fields are marked *