Databricks Data Engineer | PySpark, SQL, Python, Delta Lake & ETL/ELT

We are looking for an experienced Databricks Data Engineer to design, build, optimize, and maintain scalable cloud data pipelines. The ideal candidate has strong hands-on expertise in Databricks, PySpark, Spark SQL, SQL, Python, Delta Lake, ETL/ELT, and modern Lakehouse architecture. You will work with large datasets to build reliable batch and real-time data solutions, improve data quality, and deliver analytics-ready datasets for reporting, BI, and machine learning. Key Responsibilities Build and maintain scalable ETL/ELT pipelines using Databricks, PySpark, SQL, and Python. Develop Bronze, Silver, and Gold layers using Medallion Architecture and Delta Lake. Write optimized SQL, Spark SQL, and PySpark transformations for high-volume data. Ingest data from APIs, databases, CSV/Excel files, cloud storage, and SaaS platforms. Implement incremental loads, CDC, data validation, deduplication, error handling, and audit logging. Create and schedule Databricks Workflows, Jobs, notebooks, and automated pipelines. Use Auto Loader, Delta Live Tables (DLT), Unity Catalog, and Delta Lake optimization techniques. Design dimensional models, fact tables, dimension tables, data marts, and analytics-ready datasets. Improve Spark performance through partitioning, clustering, caching, query optimization, and file-size management. Integrate Databricks with Azure Data Factory, ADLS Gen2, Azure SQL, Synapse, Snowflake, Power BI, Event Hubs, Kafka, S3, or Redshift where required. Apply data governance, security, role-based access control, and data lineage practices. Use Git, Azure DevOps, GitHub Actions, CI/CD, and Infrastructure as Code where applicable.