Local AI Agent Platform

Summary: A Locally Hosted, Self-Managed Agentic AI Platform for Internal Workflow Automation Project Summary We are a boutique audit firm with a team of 5–7 people. We are looking for an experienced AI/automation developer to design and build a locally hosted, self-managed agentic AI platform — built on LangGraph or an equivalent framework — that connects to our internal document management and business systems to automate specific recurring workflows, starting with knowledge base agent and weekly capacity reporting. Because we handle confidential client financial information, data privacy is our top priority. Therefore, we want a solution we could host and control ourselves, sized appropriately for a small team. We are open to either an on-premises server or a rented GPU / cloud compute arrangement, provided data handling is transparent and secure. 1. Introduction & Background We are a boutique audit firm, providing audit services. Our team is small — 5 to 7 staff — which means our technology decisions need to be cost-efficient, low-maintenance, and proportionate to our size. English is the primary language used within the company and for all the documents. We are currently evaluating premium Microsoft Copilot licensing as a way to bring AI assistance into our daily workflows. However, we have two concerns: • Cost trajectory: premium per-seat AI licensing costs have been rising, and as a small firm we are concerned about unpredictable future price increases for a recurring subscription we would come to depend on. • Data privacy: as an audit firm, we handle highly sensitive client financial information. We are cautious about routing that data through third-party, multi-tenant cloud AI services, and would prefer a solution where we retain full control over where our data is processed and stored. For these reasons, we believe a locally hosted, self-managed agentic AI platform — sized for our team and built on an open, extensible framework — is likely to serve us better over the long term than an ever-scaling SaaS subscription. This RFP seeks a developer to design, build and hand over such a platform. 2. Project Objectives • Automate specific, well-defined internal workflows using AI agents, starting with the two use cases described in Section 3. • Agent Chat UI for end users (similar to Copilot or ChatGPT) • Keep firm data — particularly client-related data — under our own control at all times, rather than dependent on a third-party SaaS AI vendor. • Build on a stateful, graph-based multi-agent framework (LangGraph preferred) so the platform can be extended to new workflows over time, rather than being a one-off, single-purpose script. • Deliver a platform appropriately sized and priced for a 5–7 person team — we are not looking for enterprise-scale infrastructure. • Leave our internal Business Analyst able to understand, maintain and extend the platform after handover, supported by clear documentation. 3. Business Requirements The platform should provide one or more AI agents with controlled, permissioned access to our document management platform and relevant internal systems, able to carry out the tasks listed in below sections. The list of business systems include: - SuiteFiles (based on SharePoint) - document management - Xero Practice Management – practice management (invoices, jobs, timesheets, etc) - Audit Assistant – audit practices - DataSnipper – audit practices - Microsoft Office Suite – communication & team collaboration 3.1 Use Case — Weekly Team Capacity Review Agent An agent that runs on a weekly cadence (manually triggered or scheduled) to: • Connect to multiple internal systems to pull relevant data (e.g. timesheets, engagement schedules, staff calendars). • Consolidate that data according to pre-defined instructions we provide (e.g. how capacity/utilisation should be calculated). • Produce a structured, readable capacity report highlighting over- and under-allocated staff and upcoming workload, for review by our team leads. • Including a Chat UI for end users (similar to Copilot or ChatGPT) 3.2 Future Extensibility These two use cases are the initial build scope, but the platform is expected to serve as a reusable foundation for additional agents over time (e.g. document review support, engagement letter drafting, compliance checklist tracking). Proposals should describe how easily new agents/workflows could be added to the platform after go-live, ideally by our internal Business Analyst with light technical support. 4. Technical Requirements 4.1 Agent Orchestration Framework • LangGraph (Python) is our preferred framework, as it allows a Business Analyst — not necessarily a machine learning engineer — to build and reason about stateful, multi-agent workflows expressed as graphs. • Proposals using an equivalent framework (e.g. CrewAI, Microsoft AutoGen/Semantic Kernel, or a comparable graph/state-machine-based orchestration approach) are welcome, provided the vendor explains why it is a better fit than LangGraph for our context. • The resulting workflows should be understandable and modifiable by our internal Business Analyst post-handover, supported by clear documentation and, ideally, a readable graph/config structure rather than an opaque black box. 4.2 LLM Hosting Approach • The underlying language model(s) should be self-hosted / locally run (e.g. an open-weight model such as Llama, Qwen, Mistral or similar — vendor to recommend a model appropriate to our workload and hardware budget), rather than calling a third-party hosted AI API by default. • A hybrid approach (local model for anything touching client/firm data; external API only for clearly non-sensitive, general tasks, with our explicit sign-off) may be proposed if clearly justified and optional/toggleable. • Vendors should recommend model size(s) suitable for a 5–7 person team's concurrent usage, balancing output quality against realistic hardware/compute cost. 4.3 System Integrations • Read (and where explicitly required, write) access to our document management platform for retrieval-augmented generation (RAG) over firm documents and past quotations. • Integration with the internal systems needed for the capacity report (e.g. timesheet/practice management tool, calendars) via API, export, or another agreed method. • A permissions model that mirrors our existing access controls — agents should only see what the relevant staff member could see. 4.4 Human-in-the-Loop & Governance • Agent actions and key decisions should be logged, to support an audit trail consistent with our own profession's expectations around record-keeping. • Clear error handling: agents should flag uncertainty or missing data rather than guessing, particularly for anything client-facing. 5. Data Privacy & Security Requirements Data privacy is a critical requirement for this project, not an afterthought. As an audit firm, we hold confidential client financial information and are bound by professional confidentiality obligations. Proposals must address: • All AI processing occurring within infrastructure we control — an on-premises server or a private/dedicated cloud or rented-GPU environment — rather than shared, multi-tenant SaaS AI products. • No client or firm-confidential data being sent to third-party AI APIs by default, and never without our explicit, case-by-case approval. • Encryption of data at rest and in transit, and a sensible access-control / credentials model for any connected systems. • If any cloud or rented-GPU compute is used, clear documentation of the provider's data residency, retention policy and security posture. 6. Infrastructure Options (Including GPU Rental) We are open-minded about the underlying compute infrastructure and would like the vendor's recommendation, benchmarked against our size and privacy requirements. Two broad options should be addressed in the proposal: Option A — On-Premises Server Option B — Rented GPU / Cloud Compute Description Firm purchases and hosts a dedicated GPU-equipped computer on our premises. Compute is rented on-demand or reserved from a GPU cloud provider; no physical hardware owned by the firm. Cost profile Higher upfront capital cost; minimal ongoing compute fees. Little to no upfront cost; ongoing rental fees that scale with usage. Data control Full physical control; data never leaves our premises. Data leaves our premises to the provider — requires strong contractual and technical safeguards. Flexibility Fixed capacity; scaling up means buying more hardware. Easy to scale up/down or trial different model sizes. 7. Scope of Work & Deliverables Phase Activities Key Deliverables 1. Discovery & Design Requirements workshops; inventory of our document management and business systems/APIs; infrastructure evaluation (Section 6); privacy/security design review. Solution design document; infrastructure recommendation with cost comparison. 2. Platform Build Stand up LangGraph (or equivalent) orchestration layer; deploy self-hosted LLM; build RAG pipeline over document store; build required system integrations. Working platform with core agent infrastructure and integrations. 3. Agent Development Build the Weekly Capacity Review agent, including human-in-the-loop approval steps. One functioning agents meeting the requirements in Section 3. 4. Testing & UAT User acceptance testing with our staff; output quality/accuracy validation; security and data-handling validation. UAT sign-off; test/validation summary. 5. Deployment & Handover Production deployment; documentation of architecture and runbooks; training session(s) for our internal BA on maintaining and extending the platform. Live platform; documentation; training delivered. 6. Support (optional) Optional post-launch support/maintenance arrangement — please propose terms separately. Proposed support retainer (optional, priced separately). 8. What to Include in Your Proposal • Relevant experience: prior work with LangGraph or equivalent agent frameworks, self-hosted/open-weight LLM deployment, and retrieval-augmented generation (RAG). Experience with professional-services or regulated-industry clients is a plus. • A brief proposed technical architecture covering orchestration, model hosting, and integrations. • Your recommended infrastructure approach (on-premises vs. rented GPU, per Section 6) with reasoning and an indicative cost comparison. • Your recommended LLM(s) and why they suit our size and use cases. • A proposed timeline / milestones. • An itemised cost breakdown: build cost, any infrastructure or GPU rental costs, and optional ongoing support cost. • Portfolio examples or case studies of similar past work. • References or client feedback, if available. Not all of the above is mandatory but the more, the better. 9. Budget We have not fixed a hard budget in advance, as we want vendor proposals to reflect the most appropriate approach for the requirements above. Please provide an itemised cost proposal (build cost, infrastructure/GPU rental cost if applicable, and any optional ongoing support cost) so we can assess value for money. However, for build cost, hourly contract is preferred.