← Joburi

Senior Data Engineer Needed – Pipeline Architecture, Data Modeling & Warehouse Optimization

Buget: $50.0 - $95.0 HOURLY / FULL_TIME ⭐ 0.00 (0) United States

apache-kafka, etl-pipelines, data-warehousing, apache-airflow-platform, databricks-platform

Calificări preferate

  • Experiență: Expert
We're hiring a Senior Data Engineer to design and scale the data infrastructure behind our analytics and product decisions. You'll own our pipelines end-to-end ingestion, transformation, and serving and be the go-to for data reliability and performance. Scope of Work: - Design and maintain ETL/ELT pipelines for structured and semi-structured data - Build dimensional data models (star/snowflake schemas) for analytics use cases - Set up data quality monitoring and alerting - Optimize warehouse performance (partitioning, clustering) and manage compute costs - Build orchestration workflows (Airflow/Dagster) with retry logic and SLAs - Work with our team to translate business needs into data models Required Skills: - 5+ years data engineering experience at production scale - Expert SQL (window functions, query optimization, execution plans) - Strong Python for pipeline development - Hands-on with Snowflake, BigQuery, or Redshift - Experience with Airflow, Dagster, or Prefect - Streaming experience (Kafka/Kinesis) a plus - Cloud experience (AWS/GCP/Azure) + Terraform - dbt experience a plus How to Apply: Skills Test (please complete to be considered): To keep this efficient for both sides, please include short written answers (a paragraph or two each) to these with your proposal: 1. Query fix: Describe how you'd diagnose and fix a slow SQL query on a large table what would you check first? 2. Pipeline design: How would you design a pipeline ingesting JSON events that handles duplicates and late-arriving data, with a daily SLA? Walk through your approach and trade-offs. 3. Failure story: Tell us about a time a pipeline silently broke or loaded bad data how did you catch it, and what did you build afterward to prevent a repeat? 4. Batch vs. streaming: When would you choose batch over streaming for a new pipeline, and what would you need to know first? Candidates who skip the written answers or give generic/copy-paste responses will not be considered. We're looking for people who can show their actual reasoning.
Deschide pe Upwork

AI proposal draft

Generate a short cover letter for this job. Edit before sending.

Sign in to generate an AI proposal draft.

Autentificare