Senior Databricks Data Engineer – Hadoop Migration (1 Year contract)
Budget: $15.0 - $18.0
HOURLY / FULL_TIME
⭐ 5.00 (5)
United States
etl-pipelines, pyspark, python, databricks-platform, hadoop, etl
Qualifications préférées
- Localisation : India
- Expérience : Expert
*** YOU MUST UPLOAD YOUR RESUME IN MS WORD FORMAT TO BE CONSIDERED***
We are seeking a Senior Databricks Data Engineer for a 12-month project with a financial services client, focused on migrating an on-prem Hadoop environment to Databricks.
Start: Mid-September
Duration: 12 months
Location: Remote
Hours: Must work U.S. Eastern Time business hours (4pm - 1am IST)
Responsibilities
* Migrate data, pipelines, and workloads from Hadoop/Hive/HDFS to Databricks.
* Build and migrate pipelines using Spark/PySpark, Python, SQL, and Delta Lake.
* Refactor Hadoop workloads as needed for efficient execution on Databricks.
* Perform data validation, performance tuning, and troubleshooting.
* Implement Databricks governance and data engineering best practices.
Required Skills
* 6+ years of Data Engineering experience with strong hands-on Databricks experience.
* Strong Spark/PySpark, Python, SQL, Delta Lake, Hadoop, Hive, and HDFS skills.
* Experience with Hadoop-to-Databricks or large-scale cloud data migrations.
* Experience with newer Databricks capabilities including Unity Catalog, Lakeflow Declarative Pipelines, Lakeflow Jobs/Workflows, Auto Loader, Liquid Clustering, Serverless Compute, and Databricks SQL.
* Strong communication skills and ability to work U.S. Eastern Time business hours.
* Databricks certifications.
Nice to Have
* Financial services/banking experience.
This is a hands-on Data Engineering role focused on Hadoop-to-Databricks migration.
*** YOU MUST UPLOAD YOUR RESUME IN MS WORD FORMAT TO BE CONSIDERED***
Ouvrir sur Upwork
AI proposal draft
Generate a short cover letter for this job. Edit before sending.
Sign in to generate an AI proposal draft.
Connexion