Senior AI Voice / LiveKit Developer for Smart Intercom & Virtual Concierge Platform

Here’s an Upwork-ready job post you can use: **Senior AI Voice / LiveKit Developer for Smart Intercom & Virtual Concierge Platform** We are looking for an experienced **AI Voice / Real-Time Communications Developer** to help us build an intelligent bilingual virtual concierge and answering service for our access control and intercom platform. Our system is designed for residential communities, gated communities, condominiums, and property management companies. When a visitor arrives at an intercom, the AI should be able to speak with the visitor, identify who they are visiting, contact the resident, explain who is at the gate, understand the resident’s response, and either transfer the call, deny access, or request an authorized gate-opening action through our backend. This is not a simple chatbot project. We need someone with real experience building **real-time voice AI systems**. ### Core Responsibilities You will help design and implement: * Real-time AI voice conversations using **LiveKit** * English and Spanish conversations with automatic language detection * Speech-to-text and text-to-speech integration * LLM-based conversational logic * LiveKit Agents or equivalent real-time AI agent architecture * WebRTC audio/video sessions between: * Android intercom * AI agent * Resident mobile application * Human remote guard when escalation is required * Resident call routing and call transfer * AI function/tool calling into our backend * Visitor identification workflows * Resident lookup by apartment/property number * Visitor reason-for-entry collection * Call escalation rules * Human operator takeover * Conversation state management * Timeout, retry, no-answer, and fallback workflows * Conversation and event logging * Low-latency production deployment ### Example User Flow A visitor arrives at the intercom. The AI says: “Welcome. Who are you visiting?” Visitor: “Apartment 305.” AI: “What is your name?” Visitor: “John Smith.” The AI then contacts the resident. Resident receives a call through our mobile application. The AI says: “John Smith is at the main entrance and says he is here to visit you. Would you like to let him in?” The resident may respond: * “Yes, let him in.” * “No.” * “Let me speak with him.” * “Call my wife instead.” * “Send him to the guard.” The AI must correctly understand the intent and call the appropriate backend function. ### Important Security Requirement The AI must **never directly control a physical gate or relay based solely on its own interpretation**. The AI may interpret the resident’s response, but all access decisions must be sent to our backend API. Our backend will: 1. Verify resident identity and authorization 2. Verify the gate/door being requested 3. Validate access permissions 4. Create an audit record 5. Authorize the requested action 6. Send the command to the physical access-control device The AI layer only requests the action. ### Existing Technology Our current platform includes: * Ruby on Rails backend * Android-based intercom hardware * Native mobile applications * LiveKit/WebRTC * Cloud-based access control * Programmable relay outputs * Multi-tenant property management architecture * English and Spanish support You will work together with our existing backend and mobile developers. ### Required Experience We are specifically looking for candidates with strong experience in several of the following: * **LiveKit** * LiveKit Agents * WebRTC * Real-time voice AI * Conversational AI * OpenAI Realtime API or similar real-time LLM systems * Speech-to-text * Text-to-speech * Twilio Voice or SIP * SIP/WebRTC interoperability * Python and/or TypeScript * Function calling / tool calling * REST APIs * WebSockets * Event-driven architecture * Low-latency distributed systems Experience with Android, Kotlin, Swift, CallKit, Firebase Push Notifications, or access-control systems is a plus. ### Strong Preference We strongly prefer developers who have already built something involving: * AI phone answering * AI receptionist * Voice agent * AI call center * Virtual concierge * AI appointment agent * LiveKit voice agent * Twilio voice AI * Real-time speech application Please do not apply if your experience is primarily text-based chatbots without production voice experience. ### When Applying Please answer the following: 1. Have you built a production **real-time AI voice agent** before? 2. Have you worked directly with **LiveKit or LiveKit Agents**? 3. Which STT, TTS, and LLM providers have you integrated? 4. Have you implemented AI function/tool calling into backend APIs? 5. Have you worked with WebRTC, SIP, Twilio, or telephony systems? 6. Please provide examples, demos, GitHub repositories, or descriptions of similar projects. 7. What programming language would you recommend for the AI agent layer: Python or TypeScript, and why? 8. How would you design the system so the AI cannot accidentally authorize physical access? 9. How would you handle transferring an active visitor conversation from the AI to a resident or human guard? 10. What would you recommend for achieving natural bilingual English/Spanish conversations with low latency? ### Project Structure We expect the project to be developed in stages: **Phase 1** AI answers intercom and holds a natural visitor conversation. **Phase 2** AI connects to Vivo Control backend and looks up residents/properties. **Phase 3** AI contacts residents and interprets responses. **Phase 4** Call transfer, access authorization requests, and human-guard escalation. **Phase 5** Production hardening, monitoring, logging, security, retries, and failover. This can become a long-term project for the right developer or team. We are looking for someone who can contribute to the **architecture and engineering decisions**, not simply follow instructions.