Databricks Expert — Cost/Performance Optimization & Agentic AI Architecture

Title: Databricks Expert — Cost/Performance Optimization & Agentic AI Architecture Description: We're hiring a senior Databricks specialist to design, build, and optimize end-to-end Lakehouse solutions spanning data engineering, ML engineering, and agentic AI. You should be someone who understands real architectural trade-offs — cost vs. performance, latency vs. accuracy, flexibility vs. maintainability — and can defend every design decision with clear reasoning, not just apply defaults. What you'll do: Audit and optimize existing Databricks workloads (cluster sizing, job orchestration, Photon, autoscaling, caching) Architect end-to-end Lakehouse solutions (Delta Lake, Unity Catalog, medallion architecture) Build ML/AI pipelines using MLflow, Feature Store, and Model Serving Design agentic AI systems (Agent Bricks, DBRX) with strong token economy — efficient inference without sacrificing quality Implement Genie for natural-language querying Set up cost governance / FinOps monitoring for sustained optimization Must have: Deep hands-on Databricks Lakehouse experience DBRX and Agent Bricks experience Strong Spark performance tuning + cost optimization background Proven agentic AI/ML system design with efficient token usage Unity Catalog, Delta Live Tables, MLflow, Genie Nice to have: Databricks certifications, LLMOps/RAG experience on Databricks Please include: 2-3 examples of past Databricks cost-optimization or agentic AI work, with measurable outcomes if possible. Skills tags: Databricks, Apache Spark, Delta Lake, MLflow, Unity Catalog, DBRX, Agent Bricks, LLM, RAG, MLOps, Cost Optimization, Genie