Data Engineer for a short-term Data Migration project from AWS to Azure

Senior Data Engineer – Datamart Migration Project (Contract) Location: Offshore (India) Contract Duration: 1-2 Months (Extendable based on performance and project requirements) Employment Type: Contract / Project-Based The project involves migrating existing AWS-based data processing workloads to Azure, with a primary focus on retrofitting AWS Glue jobs (PySpark, Python, and Athena-based pipelines) into Azure Databricks environments. This role requires strong hands-on expertise in cloud data engineering, migration strategy, performance optimization, and cost-efficient data processing architectures. Key Responsibilities Analyze existing AWS Glue jobs, Athena queries, and PySpark pipelines. Retrofit and migrate AWS-based ETL/data processing workloads to Azure Databricks. Redesign and optimize data pipelines for improved scalability, performance, and cost efficiency. Work closely with client stakeholders and architecture teams to understand migration requirements. Develop and optimize PySpark-based transformations within Azure Databricks. Ensure seamless migration of datamarts, datasets, and reporting dependencies. Identify bottlenecks and recommend architectural/performance improvements. Implement best practices for monitoring, orchestration, logging, and error handling. Support testing, validation, and reconciliation of migrated workloads. Contribute to documentation and knowledge transfer activities. Technical RequirementsMandatory Skills 6+ years of experience in Data Engineering. Strong hands-on experience with: Python PySpark SQL AWS Glue Amazon Athena Strong expertise in Azure Databricks. Experience in cloud migration projects (AWS to Azure preferred). Good understanding of data warehouse/datamart architectures. Experience handling large-scale ETL/data transformation pipelines. Strong performance tuning and optimization experience. Ability to identify opportunities for reducing cloud processing costs. Experience working in Agile delivery environments. Good-to-Have Skills Experience with Delta Lake. Knowledge of Azure Data Factory (ADF). Familiarity with CI/CD pipelines for data engineering workflows. Experience with workflow orchestration tools. Exposure to Fintech, Banking, or Enterprise Data Platforms. Understanding of data governance and security best practices. Ideal Candidate Profile Strong problem-solving mindset with hands-on technical expertise. Comfortable working independently in offshore delivery models. Ability to quickly understand existing legacy pipelines and modernize them. Strong communication and stakeholder management skills. Experience delivering under tight project timelines. Project Highlights ✅ Opportunity to work on a strategic cloud modernization initiative ✅ Exposure to enterprise-scale Azure Databricks implementation ✅ Potential contract extension based on performance ✅ Collaborative international project environment