Senior Data Engineer Needed – Pipeline Architecture, Data Modeling & Warehouse Optimization
Бюджет: $50.0 - $95.0
HOURLY / FULL_TIME
⭐ 0.00 (0)
United States
apache-kafka, etl-pipelines, data-warehousing, apache-airflow-platform, databricks-platform
Предпочтительная квалификация
- Опыт: Эксперт
We're hiring a Senior Data Engineer to design and scale the data infrastructure behind our analytics and product decisions. You'll own our pipelines end-to-end ingestion, transformation, and serving and be the go-to for data reliability and performance.
Scope of Work:
- Design and maintain ETL/ELT pipelines for structured and semi-structured data
- Build dimensional data models (star/snowflake schemas) for analytics use cases
- Set up data quality monitoring and alerting
- Optimize warehouse performance (partitioning, clustering) and manage compute costs
- Build orchestration workflows (Airflow/Dagster) with retry logic and SLAs
- Work with our team to translate business needs into data models
Required Skills:
- 5+ years data engineering experience at production scale
- Expert SQL (window functions, query optimization, execution plans)
- Strong Python for pipeline development
- Hands-on with Snowflake, BigQuery, or Redshift
- Experience with Airflow, Dagster, or Prefect
- Streaming experience (Kafka/Kinesis) a plus
- Cloud experience (AWS/GCP/Azure) + Terraform
- dbt experience a plus
How to Apply: Skills Test (please complete to be considered):
To keep this efficient for both sides, please include short written answers (a paragraph or two each) to these with your proposal:
1. Query fix: Describe how you'd diagnose and fix a slow SQL query on a large table what would you check first?
2. Pipeline design: How would you design a pipeline ingesting JSON events that handles duplicates and late-arriving data, with a daily SLA? Walk through your approach and trade-offs.
3. Failure story: Tell us about a time a pipeline silently broke or loaded bad data how did you catch it, and what did you build afterward to prevent a repeat?
4. Batch vs. streaming: When would you choose batch over streaming for a new pipeline, and what would you need to know first?
Candidates who skip the written answers or give generic/copy-paste responses will not be considered. We're looking for people who can show their actual reasoning.
Открыть заказ
AI-черновик отклика
Короткий текст отклика для копирования в оффер: интерес + готовность работать.
Войдите, чтобы сгенерировать AI-черновик.
Войти