Lead Applied AI Engineer – Agentic RAG & Geospatial Copilot

Build the intelligence and conversational decision-support layer of SmartView We are developing SmartView: a vehicle-mounted urban intelligence system that transforms observations from public space into structured detections, risk assessments, recommended measures and verifiable actions. SmartView collects and combines street-level information from cameras, LiDAR, thermal imaging, GNSS and environmental sensors. Detected issues and observations are displayed geographically within our existing GIS-based platform. We are looking for a highly experienced, hands-on Lead Applied AI Engineer to design and develop the next major component of SmartView: a context-aware AI copilot that helps government and infrastructure professionals understand observations, investigate possible causes, compare interventions and arrive at a well-supported decision through conversation. This is not a standard chatbot, prompt-engineering or basic document-RAG assignment. The AI must interact with geospatial data, detection results, images, historical observations, external systems, technical standards and structured workflows. The intended user experience A user opens the SmartView map and selects a road segment, asset, detection or risk location. The AI automatically retrieves the relevant context for that location, which may include: * current detections and sensor measurements; * images and references to LiDAR or thermal observations; * previous SmartView passes at the same location; * changes and recurring patterns over time; * nearby observations and related risk indicators; * road-user behaviour and near-miss information; * asset condition and maintenance history; * applicable standards, inspection criteria and policies; * planned projects and multi-year maintenance programmes; * previously implemented measures and their measured effects. The user can then discuss the location with the AI. The AI must explain what has been observed, identify missing information, ask targeted clarification questions and distinguish between observations, assumptions and conclusions. It should propose several possible measures, explain their expected effects and trade-offs, respond to user preferences and warn when one intervention could create a new problem elsewhere. The conversation should eventually result in a structured and traceable recommendation that can be reviewed, approved and transferred to the responsible department, contractor or asset-management system. Example interaction A user selects an intersection with recurring hard-braking events and asks: Why is this location receiving a high-risk score? The AI retrieves the available evidence and explains that the score is associated with repeated braking events, restricted visibility, faded markings and frequent conflicts with delivery riders. The user then asks: Would relocating the crossing solve the problem? The AI should not immediately confirm this. It should analyse available information, identify uncertainties and potentially ask questions such as: * Is the current crossing connected to an important pedestrian route? * Are there planned works at this location? * Would relocation reduce visibility around the adjacent turn? * Is signal phasing data available? * Should the location be observed at a different time before making a decision? The AI then compares multiple interventions and produces a reasoned recommendation supported by the available evidence. Scope of work The selected specialist will be responsible for designing and implementing the AI architecture between processed SmartView data and the end-user decision-support experience. The assignment includes: 1. Architecture and context model * Analyse the current SmartView platform, data structure and intended workflows. * Design the end-to-end architecture for the AI intelligence layer. * Define how location, time, detection, asset, evidence, risk and intervention context should be represented. * Determine which information should use relational databases, PostGIS, vector search, knowledge graphs or other storage methods. * Design the separation between deterministic rules, retrieval, model reasoning and human validation. 2. Context retrieval and Agentic RAG * Build a retrieval layer that automatically assembles the correct context for a selected GIS location or detection. * Connect structured data, documents, standards, historical observations and external APIs. * Support spatial, semantic and time-based retrieval. * Ensure the AI can distinguish current information from historical or potentially outdated information. * Return source references and traceable evidence for important claims. 3. Conversational AI copilot * Develop a multi-turn conversational agent capable of maintaining location-specific context. * Implement tool/function calling for databases, GIS services, risk models and external systems. * Enable the agent to ask relevant clarification questions instead of making unsupported assumptions. * Allow the user to add operational knowledge and correct incomplete interpretations. * Maintain conversation state without contaminating unrelated locations or cases. * Support multimodal evidence, including images and structured sensor outputs. 4. Risk and measure reasoning * Connect detections and contextual factors to an explainable risk assessment. * Develop a controlled method for generating potential interventions. * Compare alternative measures and their consequences. * Present confidence, limitations, missing evidence and recommended follow-up observations. * Prevent the LLM from inventing standards, measurements or causal relationships. * Keep final decisions subject to human review and approval. The measure recommendation process should combine: * approved rules and technical standards; * retrieved project and asset information; * detection evidence and risk indicators; * comparable historical situations; * configurable decision logic; * LLM-supported analysis and explanation. 5. Structured outputs and workflow integration The AI must convert an agreed outcome into a structured action record containing, where applicable: * location and affected asset; * observed problem; * supporting evidence; * possible contributing factors; * risk level and priority; * selected or proposed measure; * responsible department or external party; * required follow-up inspection; * validation status; * proposed verification method; * links to all relevant source data. The output must be available through an API for integration into our GIS dashboard, asset-management systems and maintenance workflows. 6. Evaluation, safety and observability * Develop an evaluation framework for factual accuracy, retrieval quality and recommendation quality. * Create test scenarios for common, ambiguous and high-risk cases. * Implement logging, traceability, user feedback and audit trails. * Add guardrails for unsupported conclusions and insufficient evidence. * Monitor token usage, response time, errors and model performance. * Document how the system can be tested and improved over time. 7. Deployment and handover * Deliver production-quality, documented and maintainable code. * Containerise relevant services and support cloud deployment. * Provide API and architecture documentation. * Create technical setup and deployment instructions. * Conduct a structured handover to our existing development team. * Explain the ongoing costs, dependencies and scaling implications of the proposed architecture. Existing environment SmartView already has an existing software platform with a GIS map. The computer-vision and detection pipeline is being developed separately. This role focuses primarily on the intelligence, context, conversation and decision-support layer, but the specialist must be capable of integrating multimodal detection outputs and geospatial data. Our final technical stack is not fully fixed. We are open to a well-supported proposal. Relevant technologies may include: * Python and FastAPI; * PostgreSQL, PostGIS and pgvector; * Azure OpenAI or OpenAI APIs; * LangGraph, Semantic Kernel or comparable orchestration frameworks; * LlamaIndex or equivalent retrieval tooling; * vector databases and knowledge graphs; * Docker and cloud deployment; * REST APIs and event-driven processing; * evaluation and observability frameworks for LLM applications. We are more interested in a robust and maintainable architecture than in the use of any specific framework. Required experience You must have demonstrable experience with several of the following: * production-grade LLM applications; * agentic AI and multi-step workflows; * advanced RAG and hybrid retrieval; * tool/function calling; * conversation state and memory; * structured LLM outputs; * human-in-the-loop decision systems; * vector databases and knowledge graphs; * geospatial data or location-aware applications; * multimodal AI applications; * backend APIs and data integration; * LLM evaluation, guardrails and observability; * cloud deployment and MLOps. Experience with road infrastructure, traffic safety, GIS, asset management, digital twins or government software is highly desirable. Who we are not looking for This assignment is not suitable for someone whose experience is limited to: * writing prompts for ChatGPT; * building basic customer-service chatbots; * connecting documents to a standard RAG template; * delivering architecture advice without hands-on implementation; * producing a prototype that cannot be integrated or maintained; * relying entirely on autonomous LLM output without validation or guardrails. We need a senior engineer who can make technical decisions, write production-quality code and take ownership of a working result. Initial deliverables The first phase should produce: 1. A reviewed AI architecture and implementation plan. 2. A working context-retrieval layer using SmartView sample data. 3. A functional GIS-location-aware conversational prototype. 4. Multi-turn reasoning with clarification questions. 5. Evidence-based generation and comparison of measures. 6. Structured outputs suitable for API integration. 7. Sources, confidence indicators, guardrails and human approval. 8. An evaluation set and documented test results. 9. Deployment documentation and a roadmap towards production. A successful first phase can lead to a longer-term role in the continued development of SmartView. Confidentiality and ownership The assignment involves proprietary product concepts, datasets and workflows. The selected specialist must sign an NDA before receiving access to sensitive materials. All code, prompts, schemas, configurations, documentation and other deliverables developed under the contract must be transferred to our organisation. Project data may not be copied, retained, published or used to train unrelated models. How to apply Please include: * examples of production-grade agentic AI or advanced RAG systems you personally developed; * a clear description of your exact contribution to those systems; * examples involving tool calling, memory, structured outputs or human approval; * any experience with GIS, infrastructure, sensor data or decision-support applications; * the technology stack you would initially consider for this assignment; * your availability and expected involvement; * confirmation that you are willing to work under an NDA. Please do not submit a generic AI-generated application. We will evaluate candidates on relevant architecture choices, hands-on experience and their ability to explain complex AI systems clearly. Screening questions 1. Describe the most advanced context-aware AI agent you personally developed. What tools and data sources could it access? 2. How would you retrieve information based simultaneously on location, time, detection type and semantic relevance? 3. How would you prevent the AI from proposing measures that are unsupported by evidence or technical standards? 4. How would you combine deterministic rules, RAG and LLM reasoning in a single decision-support workflow? 5. How would you maintain conversation memory for one road location without allowing information from another case to contaminate the answer? 6. What methods would you use to evaluate whether the AI’s conclusions and recommendations are reliable? 7. Which parts of this system would you build yourself, and for which parts would you recommend an additional specialist? ⸻ Kortere Upwork-variant Titel Senior AI Engineer – Geospatial Copilot, Agentic RAG & Decision Support Korte opdrachtomschrijving We are developing SmartView, a vehicle-mounted urban intelligence system that transforms camera, LiDAR, thermal, environmental and geospatial observations into detections, risks and actionable measures. We are looking for a senior, hands-on Applied AI Engineer to build a context-aware conversational copilot inside our existing GIS platform. When a user selects a road segment, asset or detection on the map, the AI must automatically retrieve all relevant context: current observations, historical passes, nearby risks, sensor evidence, images, applicable standards, maintenance history and planned projects. The AI must then conduct an intelligent, multi-turn conversation. It should explain the evidence, identify missing information, ask clarification questions, compare possible causes, propose alternative measures and discuss their effects and trade-offs. The conversation must result in a structured, traceable recommendation that can be reviewed and transferred to the responsible department or asset system. This is not a basic chatbot or document-RAG assignment. We require experience with: * agentic AI and multi-step workflows; * advanced RAG and hybrid retrieval; * tool/function calling; * conversation state and memory; * structured outputs and human approval; * geospatial or location-aware data; * LLM evaluation, citations and guardrails; * backend APIs and production deployment. Experience with GIS, infrastructure, road safety, multimodal data or government decision-support software is highly desirable. The first phase includes the architecture, a working GIS-aware conversational prototype, evidence-based measure generation, structured API outputs, evaluation and deployment documentation. Please share relevant systems you personally built, describe your exact contribution and explain how you would prevent unsupported or hallucinated recommendations. An NDA and full transfer of code and intellectual property are required. A successful first phase can lead to a long-term role within SmartView.