Backend Engineer – Real-Time AI Avatar Video Integration (WebRTC, FastAPI, Celery)

We're building an AI powered coaching app. Our backend runs a deterministic AI pipeline, LangGraph, Celery, vector DB, that generates text responses. We're now adding a bidirectional video layer: our AI responds via a talking avatar, and can also receive and analyze user submitted video input. Scope of work: Integrate a real time avatar streaming API. Candidates include HeyGen LiveAvatar, Anam, Tavus, and D-ID. You will help us evaluate and recommend one to generate our AI's video responses from text, capped at 60 seconds per response. Build inbound pipeline: user video, then frame and audio extraction, then transcription, then emotion and expression analysis using facial and prosody signals, then structured JSON output with segmented, timestamped data and confidence scores, feeding our existing pipeline. Manage full WebRTC session lifecycle including signaling, ICE and SDP, auth, concurrency, and teardown, server side. Chain the multi step flow, extract, analyze, generate, stream, via Celery with timeouts and graceful fallback to text on any failure. Enforce latency targets, around 180ms candidate response latency and a 60 second hard cap, server side. Document a clean API contract for our separate mobile team to consume. You are not building mobile app UI.