← Jobb

Hands-on Apache Airflow / Astronomer Astro Mentor End-to-End Development & Git Workflow

Budget: $15.0 - $45.0 HOURLY / PART_TIME ⭐ 0.00 (0) India

snowflake, python, apache-airflow-platform, git

Föredragna kvalifikationer

  • Erfarenhet: Expert
We are looking for an experienced Apache Airflow / Astronomer Astro developer to provide practical, hands-on mentoring for a developer who is new to the Airflow/Astro development lifecycle. This is not a request for someone to simply build a pipeline for us, and we are not looking for a theoretical Airflow training course. The objective is to conduct a few interactive online screen-sharing sessions where the mentor guides the developer through the complete development lifecycle by building a simple vanilla Airflow pipeline together. Developer Background The developer has: Good knowledge of Snowflake Decent working knowledge of Python Limited experience with Git/GitHub-style development workflows No significant previous hands-on experience developing and delivering Airflow/Astro pipelines Therefore, we need someone who is comfortable explaining the practical development workflow from the beginning rather than assuming strong Git/DevOps experience. Hands-on Objective By the end of the sessions, the developer should have personally completed an end-to-end exercise similar to: Source/File → Airflow DAG → simple Python transformation → Snowflake The exact pipeline can be kept deliberately simple. The purpose is to learn the development and delivery process rather than build a complex data-engineering solution. Areas We Want Covered The mentor should guide the developer hands-on through: Development environment setup Minimum software required VS Code/IDE setup Python and virtual environments Git Astro CLI/Airflow setup Explain whether Docker is required and, where practical, demonstrate a minimal development setup without unnecessary tooling Git and repository workflow Clone a repository Understand the basic repository structure Create a feature branch Make code changes Git status/diff Commit changes Pull/rebase as appropriate Push the branch Create a Pull Request Explain code review and merge workflow Demonstrate how this typically works in a real enterprise development team Create a basic Astro/Airflow project Understand the project/folder structure DAG folder Dependencies/requirements Configuration Connections and variables Basic testing structure Develop a vanilla Airflow DAG Create a simple DAG from scratch Create multiple tasks Python task/operator Task dependencies Parameters/configuration Basic error handling Logging Scheduling Manually trigger and rerun the DAG Snowflake integration Configure a DEV/test Snowflake connection securely Do not hard-code credentials Read/write a small sample dataset Execute a simple transformation/query Understand how Airflow connections/secrets are normally handled Run and troubleshoot Run the DAG Use the Airflow UI Understand DAG/Graph/Grid views Inspect task execution Read logs Identify and fix a deliberately introduced simple failure Rerun failed tasks Development-to-deployment lifecycle Explain and, where possible, demonstrate the complete flow: Requirement → Repository → Branch → Develop DAG → Local/Test Environment → Test → Commit → Push → Pull Request → Review → Merge → CI/CD → DEV Airflow/Astro deployment We particularly want the developer to understand which activities happen locally versus Git versus CI/CD versus the Airflow/Astro environment. Enterprise development concepts A practical introduction to: DEV/UAT/PROD separation CI/CD Environment-specific configuration Secrets management Service accounts Package/dependency management Basic testing Branching/PR practices What access and software are normally required on a corporate-managed laptop Preferred Training Approach This should be highly hands-on. We would prefer the developer to share her screen and perform the activities herself, with the mentor providing instructions and troubleshooting, rather than the mentor simply sharing their screen and demonstrating everything. The objective is that after the engagement she can repeat the process independently. We are happy to use a completely new/sample Git repository, sample data and a test/free environment where possible. No company or production systems need to be accessed. Expected Outcome At completion, the developer should: Have a working vanilla Airflow/Astro pipeline Understand the Astro/Airflow project structure Be able to make a DAG change independently Be comfortable with the basic Git branch/commit/push/PR workflow Know how to run and troubleshoot a DAG Understand how Snowflake connections should be configured Understand the basic CI/CD and deployment lifecycle Understand the minimum tools/access she would need when joining a real enterprise Airflow development project Engagement We expect this to be a short mentoring engagement conducted through multiple online screen-sharing sessions, rather than a long formal training program. Initial mentoring expected to be 4-6 hours. It can be extended.
Öppna på Upwork

AI proposal draft

Generate a short cover letter for this job. Edit before sending.

Sign in to generate an AI proposal draft.

Logga in