Applied RAG & Agentic AI Inside Enterprise Delivery

Keep the base model unchanged and inject relevant, retrieved context at query time from an index built over the organisation's own documents.

Fine-tuning a model on internal architecture documentation.

Enterprise architecture knowledge changes weekly; fine-tuning bakes a snapshot into weights and gives you no way to cite a source or retract a superseded decision. Retrieval keeps knowledge in a system you can update, audit and permission — and re-indexing a corrected ADR takes minutes.

Retrieval quality becomes the ceiling on answer quality, so the hard engineering moves into chunking, embeddings and ranking. Every query pays retrieval latency and spends context window on retrieved text.

Split along real document boundaries — an ADR's decision and consequences, an API contract's endpoint — and attach metadata (source, system, version, status, date) to every chunk so retrieval can filter, not just similarity-match.

Fixed-size character windows with naive overlap.

Fixed windows sever decisions from their rationale, which is precisely the pairing that makes an ADR useful. Status metadata is what lets the system prefer current decisions over superseded ones — without it, retrieval will happily surface a reversed decision because it is textually similar.

Structure-aware parsing per document type is real work and breaks when authors deviate from templates. It also creates a dependency on documentation hygiene that the platform cannot itself enforce.

Every answer returns its supporting sources, and the assistant declines rather than speculating when retrieval finds nothing relevant. Output is a draft for an architect to accept, never an authority.

A fluent chat interface that always produces an answer, with sources optional.

Trust is the adoption bottleneck, not capability. Engineers extend trust to a tool they can verify in one click and abandon one that has confidently misled them once. "I don't have a grounded answer" is a feature.

Lower apparent coverage — the system visibly refuses questions a less careful tool would answer — and that reads as weakness to anyone judging on demo polish rather than reliability.

Apply the capability to the delivery workflow — design review, code and test scaffolding, documentation — rather than placing an LLM inside a customer-facing or money-moving request path.

Embedding an LLM in a runtime application or money-moving request path. That option was evaluated for a customer-facing assistant and deferred: non-determinism and variable latency fail regulated SLAs, and a fluent wrong answer in a transaction path is an incident, not a draft.

Non-determinism and variable latency are acceptable when a human reviews the output before it has effect. They are not acceptable in a regulated transaction path, where you need bounded latency and reproducible behaviour.

The value shows up as engineering leverage rather than a headline product feature, which makes it harder to attribute and harder to fund.