Hands-on Apache Airflow / Astronomer Astro Mentor End-to-End Development & Git Workflow
Rozpočet: $15.0 - $45.0
HOURLY / PART_TIME
⭐ 0.00 (0)
India
snowflake, python, apache-airflow-platform, git
Preferované kvalifikácie
- Skúsenosť: Expert
We are looking for an experienced Apache Airflow / Astronomer Astro developer to provide practical, hands-on mentoring for a developer who is new to the Airflow/Astro development lifecycle.
This is not a request for someone to simply build a pipeline for us, and we are not looking for a theoretical Airflow training course.
The objective is to conduct a few interactive online screen-sharing sessions where the mentor guides the developer through the complete development lifecycle by building a simple vanilla Airflow pipeline together.
Developer Background
The developer has:
Good knowledge of Snowflake
Decent working knowledge of Python
Limited experience with Git/GitHub-style development workflows
No significant previous hands-on experience developing and delivering Airflow/Astro pipelines
Therefore, we need someone who is comfortable explaining the practical development workflow from the beginning rather than assuming strong Git/DevOps experience.
Hands-on Objective
By the end of the sessions, the developer should have personally completed an end-to-end exercise similar to:
Source/File → Airflow DAG → simple Python transformation → Snowflake
The exact pipeline can be kept deliberately simple. The purpose is to learn the development and delivery process rather than build a complex data-engineering solution.
Areas We Want Covered
The mentor should guide the developer hands-on through:
Development environment setup
Minimum software required
VS Code/IDE setup
Python and virtual environments
Git
Astro CLI/Airflow setup
Explain whether Docker is required and, where practical, demonstrate a minimal development setup without unnecessary tooling
Git and repository workflow
Clone a repository
Understand the basic repository structure
Create a feature branch
Make code changes
Git status/diff
Commit changes
Pull/rebase as appropriate
Push the branch
Create a Pull Request
Explain code review and merge workflow
Demonstrate how this typically works in a real enterprise development team
Create a basic Astro/Airflow project
Understand the project/folder structure
DAG folder
Dependencies/requirements
Configuration
Connections and variables
Basic testing structure
Develop a vanilla Airflow DAG
Create a simple DAG from scratch
Create multiple tasks
Python task/operator
Task dependencies
Parameters/configuration
Basic error handling
Logging
Scheduling
Manually trigger and rerun the DAG
Snowflake integration
Configure a DEV/test Snowflake connection securely
Do not hard-code credentials
Read/write a small sample dataset
Execute a simple transformation/query
Understand how Airflow connections/secrets are normally handled
Run and troubleshoot
Run the DAG
Use the Airflow UI
Understand DAG/Graph/Grid views
Inspect task execution
Read logs
Identify and fix a deliberately introduced simple failure
Rerun failed tasks
Development-to-deployment lifecycle
Explain and, where possible, demonstrate the complete flow:
Requirement → Repository → Branch → Develop DAG → Local/Test Environment → Test → Commit → Push → Pull Request → Review → Merge → CI/CD → DEV Airflow/Astro deployment
We particularly want the developer to understand which activities happen locally versus Git versus CI/CD versus the Airflow/Astro environment.
Enterprise development concepts
A practical introduction to:
DEV/UAT/PROD separation
CI/CD
Environment-specific configuration
Secrets management
Service accounts
Package/dependency management
Basic testing
Branching/PR practices
What access and software are normally required on a corporate-managed laptop
Preferred Training Approach
This should be highly hands-on.
We would prefer the developer to share her screen and perform the activities herself, with the mentor providing instructions and troubleshooting, rather than the mentor simply sharing their screen and demonstrating everything.
The objective is that after the engagement she can repeat the process independently.
We are happy to use a completely new/sample Git repository, sample data and a test/free environment where possible. No company or production systems need to be accessed.
Expected Outcome
At completion, the developer should:
Have a working vanilla Airflow/Astro pipeline
Understand the Astro/Airflow project structure
Be able to make a DAG change independently
Be comfortable with the basic Git branch/commit/push/PR workflow
Know how to run and troubleshoot a DAG
Understand how Snowflake connections should be configured
Understand the basic CI/CD and deployment lifecycle
Understand the minimum tools/access she would need when joining a real enterprise Airflow development project
Engagement
We expect this to be a short mentoring engagement conducted through multiple online screen-sharing sessions, rather than a long formal training program.
Initial mentoring expected to be 4-6 hours. It can be extended.
Otvoriť na Upwork
AI proposal draft
Generate a short cover letter for this job. Edit before sending.
Sign in to generate an AI proposal draft.
Prihlásiť