Self-hosting an enterprise RAG platform means running a handful of well-understood components — a relational database, a cache, a vector store and an application layer, with optional local models — on infrastructure you control. It is more operational work than signing up for a hosted service, and less than most teams fear. This is what the architecture looks like and what running it actually involves.
The reference architecture
RAGSuite is built from components your platform team already knows how to operate. Nothing here is exotic — that is the point: sovereignty should not require a bespoke stack only one vendor can run.
The model layer is the only real choice point: connect a hosted provider (OpenAI, Anthropic, Mistral, Gemini) or run local models via Ollama, in which case inference happens inside your network and nothing leaves it.
How a request flows
It helps to trace a single answer end to end. Every answer is therefore traceable back to the documents that grounded it — by design, not as an add-on.
Ingest → retrieve → ground → cite
- Ingest — AI Connectors bring content in (crawl, upload, Gmail, MCP, n8n (Beta)); text is chunked and embedded into the vector store, metadata into PostgreSQL.
- Retrieve — the query is embedded and matched against the vector store to pull the most relevant passages.
- Ground & generate — the passages are passed to the chosen model, which composes an answer constrained to those sources.
- Cite & log — the answer returns with citations to the source passages, and the interaction is recorded for audit and the quality loop.
Deployment shapes
The same architecture runs in three postures, depending on how strict your requirements are.
| Posture | Where it runs | Best for |
|---|---|---|
| On-premise | Entirely within your data centre | KRITIS, classified, strict residency |
| Private cloud | Your own tenant, EU-resident if required | Most regulated enterprises |
| Air-gapped | No outbound connectivity at all | The most sensitive workloads |
Because Enterprise licensing is an offline key (a signed token), an air-gapped deployment is a first-class scenario, not a workaround.
The operational realities — honestly
On sizing: the database, cache and vector store are modest for typical enterprise knowledge bases and scale predictably; the variable cost is inference. Route to a hosted model and your local footprint stays small; run models locally for full isolation and you provision for that workload. Compare Models makes it straightforward to test the quality-versus-cost trade per project before you commit hardware. For the full picture, see the platform page and the self-hosted vs. hosted comparison.
Frequently asked questions
How much of a team does this need?
If you already run containerised services with a database and cache, self-hosting RAGSuite fits within those existing capabilities. The new muscle is mostly around the vector store and, optionally, model inference — both well-trodden ground in 2026.
Do we have to run our own models?
No. Local models via Ollama are the path to full network isolation, but you can route to a hosted provider and keep the deployment lean. The platform is model-agnostic either way, so you can start hosted and move local later without re-architecting.
Can we start small and grow?
Yes. The Community Edition is a legitimate way to validate the architecture on a single project, and the same components scale to a full Enterprise deployment.
Sources & further reading
- Ollama — run open models locally — local inference inside your network
- pgvector — vector similarity search for PostgreSQL — the embeddings index
- RAGSuite — platform overview — the full pipeline, API-first