Astronomer/Airflow Architect

Job Description

<p><b>Role: Astronomer/Airflow Architect</b></p><p><b>Location: Mississauga, ON (Hybrid)</b></p><p><br></p><p><b>The Solutions Architect specializing in Cloud Orchestration, will work with a small group of technologists and become a trusted advisor delivering technical solutions leveraging Apache Airflow hosted on the Astronomer platform. In this role, he will engineer a wide range of use cases where, Apache Airflow sits at the center of the solution, with the goal of rapidly developing solutions for the use cases. </b></p><p><br></p><ul><li><b>Help guide a small group of technologists in their Apache Airflow journeys, including identifying new use cases and onboarding new domain teams.</b></li><li><b>Review, optimize, and tune data ingestion and extraction pipelines orchestrated by Airflow.</b></li><li><b>Development of frameworks to manage and engineer a large number of pipelines for the entire domain.</b></li><li><b>Build architecture, data flow, and operational diagrams and documents, with detailed physical and logical layers.</b></li><li><b>Provide reference implementations of various activities, including composing data pipelines in Airflow, implementing new Airflow features, or integrating Airflow with 3rd party solutions.</b></li><li><b>Keep up with the latest Astro and Airflow features, in order to better advise the technologies on impactful improvements they can make.</b></li><li><b>Collaborate to build reusable assets, automation tools, and documented best practices.</b></li><li><b>Interact with Domain and Engineering teams to channel product feedback and requirements discussions.</b></li><li><b>Work with technology team members to ensure that are realizing value in their Airflow and Astronomer journeys</b></li></ul><p><b>Skills Required:</b></p><ul><li><b>Experience with Apache Airflow in production environments.</b></li><li><b>Experience in designing and implementing ETL, Data Warehousing, and ML/AI use cases.</b></li><li><b>Proficiency in Python.</b></li><li><b>Knowledge of Azure cloud-native data architecture. </b></li><li><b>Demonstrated technical leadership on team projects.</b></li><li><b>Strong oral and written communication skills.</b></li><li><b>Willingness to learn new technologies and build reference implementations.</b></li><li><b>Experience in migrating workflows from legacy schedulers (Tidal) to Apache Airflow.</b></li><li><b>Experience in integrating with Azure Data Factory pipelines.</b></li><li><b>Experience </b></li><li><b>Snowflake and Databricks experience.</b></li><li><b>Experience working with containers.</b></li><li><b>SQL experience.</b></li><li><b>Kubernetes experience, either on-premise or in the cloud.</b></li><li><b>Enterprise data experience in regulated environments.</b></li></ul><p></p>