All case studies
Case Study

Designing an Enterprise GenAI Assistant with RAG and Human-in-the-Loop Governance

Identity-aware retrieval, an honest comparison against plain search, and where autonomy should stop

AI ArchitectureHypothetical21 min readPublished 20 Aug 2026Updated 21 Aug 2026

Hypothetical. A constructed scenario used to work through an architecture problem.

GenAILLMRAGCopilotAzure AIMicrosoft 365Microsoft Entra IDMicrosoft GraphVector SearchAI Governance

Executive Summary

The case. An enterprise wants an internal assistant that answers questions from its own documentation, policies and project material held in Microsoft 365.

The problem being investigated. Whether generative AI is the right instrument at all, and if so, which architecture — retrieval-augmented generation, a fine-tuned model, or an agentic system that also acts — carries an acceptable risk profile for content that is confidential and permission-scoped.

Why it matters. The failure mode is not a wrong answer. It is a confident wrong answer, or a correct answer assembled from a document the asker was never entitled to read. The second is a data breach with good grammar.

The decision involved. This study recommends identity-aware RAG with human-in-the-loop governance, rejects fine-tuning on proprietary content as the primary mechanism, and treats agentic autonomy as a later, separately-justified decision rather than a natural extension.

Disclosure. This is a hypothetical study — a constructed scenario used to work through an architecture problem. It is not a delivered system. Every figure in Business Impact is labelled Estimated, Projected or Illustrative; none is Measured, and none describes a production result.

01

Context

The organization holds most of its institutional knowledge in Microsoft 365: policy documents, architecture notes, project records and long email and chat threads. Search works, in the narrow sense that it returns documents. People still ask colleagues, because a returned document is not an answer.

Business context

The pressure to "add AI" is real and mostly healthy. The risk is that it arrives as a product decision rather than an architecture decision, and the permission model is discovered afterwards.

Technology context

Content is permission-scoped through Microsoft Entra ID and SharePoint. Any retrieval system that ignores that scoping does not have a tuning problem; it has a disclosure problem.

What is fact and what is assumption

  • FACT. Microsoft 365 content carries per-item permissions resolvable through Microsoft Graph and Entra ID.
  • FACT. Azure AI Search supports security trimming so results can be filtered by the caller's identity (see References).
  • ASSUMPTION. The corpus is large enough that retrieval quality, not model capability, is the binding constraint.
  • ASSUMPTION. Users will accept a cited answer more readily than an uncited one.
  • HYPOTHETICAL. The organization, its corpus size and its usage patterns. No real tenant is described.
02

Problem Statement

People cannot get answers from content they are entitled to read, and any system that fixes that naively will also surface content they are not.

Who is affected

  • Knowledge workers lose time reconstructing answers that exist somewhere in the corpus.
  • Subject matter experts become human search engines and are interrupted constantly.
  • Security and compliance must be able to explain, after the fact, why a given answer was produced and what it drew on.

Current limitations

  • Keyword search returns documents, not answers, and ranks by relevance rather than by currency — superseded policy outranks the version that replaced it.
  • Answers require synthesis across several documents, which is exactly what search does not do.
  • There is no audit trail connecting a question to the material used to answer it.

Consequences

Technically, institutional knowledge is present but not addressable. Organizationally, expertise is a bottleneck. The temptation is to solve this with a general-purpose assistant pointed at everything — which converts a retrieval problem into a disclosure problem.

03

Objectives

Framed so each could be argued as met or not met.

  • Return synthesized answers with citations the asker can open and verify
  • Never surface content the asker could not already open directly — retrieval must be identity-aware, not filtered after the fact
  • Make every answer auditable: question, retrieved sources and response retained for review
  • Keep a human in the loop wherever an answer would otherwise become an action
  • Prefer measurable groundedness over fluency; a confident wrong answer is worse than an admission of uncertainty
  • Avoid architectural lock-in to a single model vendor
  • Establish an evaluation baseline before rollout, so quality claims are testable
04

Current State

How a question is answered today, and where the time goes.

StepCurrent mechanismFailure mode
Locate candidate materialKeyword search across SharePoint and TeamsRecall is acceptable, precision is poor; duplicates and superseded versions rank alongside current ones
Judge currencyManual — open the document and lookNo reliable signal distinguishing current policy from a superseded draft
SynthesizeHuman reads several documentsSlow; quality varies with the reader's familiarity
VerifyAsk a subject matter expertExpert becomes a bottleneck; answer is not recorded anywhere
RecordUsually not recordedThe same question is answered repeatedly with no accumulation

Measurement note. A real engagement would instrument this with a task-based baseline before building anything. That baseline does not exist here, which is precisely why every figure in Business Impact is labelled as an estimate.

05

AI Opportunity

This is a genuinely good fit for generative AI, and it is worth being precise about why: the corpus already contains the answers, and the missing capability is synthesis with attribution. That is what a language model over retrieved context does well.

It is equally worth being precise about where it is a poor fit. Anything requiring a guarantee — an entitlement decision, a regulatory interpretation, an irreversible action — should not be delegated to a probabilistic system, however good the retrieval.

AI requires human assistance

Where AI creates value

  • Synthesizing an answer across several documents, with citations the reader can open
  • Surfacing the current version of a policy and naming what it superseded
  • Drafting a first response for a subject matter expert to correct rather than compose
  • Making tacit knowledge in long threads addressable through natural language

What should not be automated

  • Deciding entitlement — who may see what is an authorization question, not a retrieval one
  • Regulatory or contractual interpretation presented as authoritative
  • Any irreversible action taken without human confirmation
  • Answering when retrieval returned nothing relevant — abstention must be a supported outcome, not a fallback to fluent invention
06

Alternatives

Three architectures compared. Option A is not a straw man — for a meaningful class of questions it remains the correct answer, and the design keeps it available.

Option A

Traditional search / knowledge portal

Improve the existing search experience: better metadata, curated landing pages, promoted results, explicit ownership of canonical documents.

Advantages

  • Deterministic and explainable
  • No hallucination risk whatsoever
  • Permission model already correct and battle-tested
  • Cheapest to run
  • Users can verify by reading the source directly

Disadvantages

  • Returns documents rather than answers
  • No synthesis across sources
  • Quality depends on metadata discipline that erodes
  • Does not address the expert bottleneck
Cost: LowRisk: LowScalability: GoodComplexity: Low

Option B

Identity-aware RAG assistant

Recommended

Retrieve permission-trimmed passages for the asking user, then have a language model synthesize an answer strictly grounded in those passages, with citations.

Advantages

  • Answers rather than links
  • Grounded in retrieved content so claims are traceable
  • No model retraining when content changes
  • Permission trimming can be enforced at retrieval
  • Model vendor can be swapped

Disadvantages

  • Hallucination risk is reduced but never eliminated
  • Answer quality is bounded by retrieval quality
  • Adds meaningful operational surface — index
  • embeddings
  • evaluation
  • Latency and cost per question
Cost: MediumRisk: MediumScalability: GoodComplexity: Medium

Option C

Agentic architecture that also acts

An assistant that plans multi-step tasks and invokes tools — raising tickets, updating records, sending messages — with limited human intervention.

Advantages

  • Handles multi-step work end to end
  • Highest ceiling on time saved
  • Can close the loop rather than just informing it

Disadvantages

  • Risk stops being linear — errors compound across steps
  • Prompt injection becomes an execution vector rather than a disclosure one
  • Attribution of a bad outcome to a step is hard
  • Requires mature observability that does not yet exist here
  • Blast radius of a wrong decision is unbounded
Cost: HighRisk: HighScalability: Unproven at this maturityComplexity: High
07

Proposed Solution

Identity-aware retrieval-augmented generation, with abstention as a first-class outcome and a human in the loop wherever an answer would become an action.

How a question is answered

  1. The user authenticates; the request carries their identity, never a service identity acting on their behalf.
  2. Retrieval runs against an index that is trimmed to what that identity may read, before ranking — not filtered afterwards.
  3. Top passages are assembled into context with their provenance attached.
  4. The model answers strictly from that context and cites each claim.
  5. If retrieval returned nothing sufficiently relevant, the system says so instead of answering.
  6. Question, retrieved source identifiers and response are written to an audit trail.

Where the human stays

The assistant informs; it does not decide or act. Any transition from answer to action — raising a ticket, changing a record, notifying someone — requires explicit human confirmation. That boundary is the main thing separating Option B from Option C, and it is deliberate rather than temporary.

08

Solution Architecture

The security-relevant property of this architecture is that authorization is applied at retrieval, in the user’s own identity. Everything else is replaceable.

Client experience

Presents the answer with citations and an explicit uncertainty state. Never renders an answer without its sources; abstention is displayed as a legitimate result rather than an error.

Orchestration service

Runs the retrieve-then-generate pipeline, enforces prompt construction, applies the abstention threshold, and writes the audit record. The only component that talks to the model.

Identity-aware retrieval (Azure AI Search)

Hybrid keyword and vector retrieval with security trimming applied as a filter on the caller’s group membership, so unauthorized documents are never candidates.

Embedding and indexing pipeline

Chunks documents, generates embeddings, and carries each chunk’s ACL and version metadata into the index. Re-indexes on change so superseded content can be demoted.

Microsoft Graph connector

Source of content and of the permission metadata that makes trimming possible. Permissions are ingested with the content, not inferred later.

Model gateway (Azure AI)

Abstracts the model behind one interface so a vendor can be replaced. Applies content filtering and enforces per-tenant rate limits.

Evaluation harness

Scores groundedness, citation correctness and abstention rate against a fixed question set on every change. Without it, "the answers got worse" is unfalsifiable.

Audit store

Retains question, retrieved source identifiers, model version and response for review. Stores identifiers rather than document bodies.

Data flow

A user question arrives with the user’s identity token. The orchestration service resolves group membership and issues a hybrid retrieval query with a security filter, so the candidate set contains only chunks the user may read. Retrieved chunks and their provenance are assembled into a prompt with instructions to answer only from the supplied context and to abstain otherwise. The model gateway forwards the request, applies content filtering and returns a completion. The orchestrator validates that every claim carries a citation, applies the abstention threshold, returns the answer with links, and writes question, source identifiers, model version and response to the audit store. Indexing runs separately: the Graph connector ingests content and its ACLs, the pipeline chunks and embeds it, and the index is updated on change.

Integrations

  • Microsoft Graph for content ingestion and permission metadata
  • Microsoft Entra ID for authentication and group membership resolution at query time
  • Azure AI Search for hybrid keyword plus vector retrieval with security trimming
  • Azure AI model gateway for completions and content filtering
  • SIEM or log analytics for the audit trail and anomaly detection

Security boundaries

  • Retrieval executes in the asking user’s identity — the assistant holds no standing read access to the corpus
  • ACLs are ingested with content and applied as a pre-ranking filter, never as post-hoc redaction
  • Retrieved content is treated as untrusted input to the model, which is the mitigation for indirect prompt injection
  • The model gateway is the only egress path to a model provider, so data flow has one auditable choke point
  • The audit store holds source identifiers rather than document bodies, so it does not become a second uncontrolled copy of the corpus

Human in the loop

  • Any transition from answer to action requires explicit human confirmation
  • Low-confidence or abstained answers route to a subject matter expert rather than being guessed
  • Sampled answers are reviewed for groundedness on a defined cadence, not only when someone complains
  • Corpus scope changes — adding a new content source — are a governance decision with a named approver
09

Technology Stack

Chosen so that the model is the most replaceable component, not the least.

AI

Azure AILLMGenAICopilotRAG

Retrieval

Azure AI SearchVector SearchHybrid rankingSemantic ranking

Content

Microsoft 365SharePointMicrosoft Graph connectors

Identity

Microsoft Entra IDGroup-based authorizationSecurity trimming

Governance

AI GovernanceContent filteringEvaluation harnessAudit logging

Operations

ObservabilityCost controlsRate limitingModel version pinning
10

Architecture Decisions

Four decisions, each recorded with what it costs.

Decision 01

Use retrieval-augmented generation rather than fine-tuning on proprietary content

The corpus changes continuously and is permission-scoped per item. The requirement is current, attributable answers.

Alternatives considered

  • Prompt engineering over a general model
  • Retrieval-augmented generation
  • Fine-tuning on the corpus
  • Continued pre-training

Reason

Fine-tuning bakes content into weights, which cannot be permission-trimmed per user, cannot be updated when a document changes, and cannot cite a source. Those three defects are disqualifying here regardless of answer quality.

Benefits

  • Content stays in a permissioned store
  • Citations are possible
  • Updates are an index operation not a training run
  • Model vendor stays replaceable

Trade-offs

  • Answer quality is bounded by retrieval quality
  • Per-question latency and cost
  • Substantial indexing and evaluation machinery to operate

Risks

  • Poor chunking silently degrades every answer
  • Index and source drift out of sync
  • Retrieval becomes the unmonitored bottleneck

Decision 02

Apply authorization trimming at retrieval, in the user identity, rather than filtering after generation

The corpus is permission-scoped. A model that has seen a passage can paraphrase it even if the passage is later withheld.

Alternatives considered

  • Post-generation redaction
  • Separate index per security group
  • Query-time security filter on an ACL-carrying index

Reason

Once unauthorized content enters the context window it has already leaked — redaction afterwards is theatre. A per-group index multiplies cost and drifts. A query-time filter on ingested ACLs keeps one index correct by construction.

Benefits

  • Unauthorized content is never a retrieval candidate
  • One index to operate
  • Permission changes take effect on re-index
  • Auditable in the user identity

Trade-offs

  • ACLs must be ingested and kept fresh
  • Group membership resolution adds query latency
  • Stale ACLs become a correctness bug rather than a performance one

Risks

  • ACL ingestion lag creating a window of over-permissive retrieval
  • Complex nested group structures resolving incorrectly

Decision 03

Treat abstention as a first-class outcome with an explicit confidence threshold

A language model asked a question with irrelevant context will still produce fluent text. Fluency is not grounding.

Alternatives considered

  • Always answer
  • Answer with a confidence caveat
  • Abstain below a retrieval-relevance threshold

Reason

The expensive failure is a confident wrong answer, because it is acted upon. Abstention converts that into a visible gap a human can fill.

Benefits

  • Reduces the most damaging failure mode
  • Makes retrieval quality measurable through abstention rate
  • Builds trust — a system that says "I don’t know" is believed when it does answer

Trade-offs

  • Users perceive abstention as failure unless the experience frames it well
  • Threshold tuning is empirical and ongoing
  • Higher abstention rate looks worse on naive usage dashboards

Risks

  • Threshold set to please a dashboard rather than to protect correctness
  • Abstention masking a retrieval defect nobody investigates

Decision 04

Defer agentic autonomy to a separate decision rather than treating it as a roadmap increment

Agentic architectures are an obvious next step and are frequently adopted as one.

Alternatives considered

  • Build agentic from the start
  • Add tool-calling incrementally
  • Ship read-only and revisit autonomy as its own decision

Reason

Autonomy changes the risk class. With retrieval only, a prompt injection discloses; with tool access, it acts. That is not an increment on the same curve, and the observability required to detect a bad multi-step action does not yet exist here.

Benefits

  • Bounded blast radius
  • Simpler failure analysis
  • Observability can mature before autonomy depends on it
  • Trust established before stakes are raised

Trade-offs

  • Leaves multi-step time savings unrealised
  • May be perceived as unambitious
  • A second approval cycle later

Risks

  • Organizational pressure to add tool access informally
  • Scope creep through connectors that quietly act
11

Implementation Approach

Sequenced so that evaluation exists before scale, not after.

  1. 01Discovery and evaluation baseline

    Weeks 1–4

    Agree the initial corpus scope, confirm ACL availability through Graph, and build a fixed question set with expert-approved answers. This set is the only thing that will later make "it got better" a testable claim.

    Milestones

    • Corpus scope agreed
    • ACL ingestion proven
    • Evaluation question set approved by experts

    Success measures

    • Question set size
    • Percentage of corpus with resolvable ACLs

  2. 02Retrieval prototype

    Weeks 4–10

    Build indexing, chunking and hybrid retrieval with security trimming. Deliberately no generation yet — retrieval quality is the binding constraint and is easier to measure alone.

    Milestones

    • Index populated with ACLs
    • Hybrid retrieval tuned
    • Trimming verified against known permission cases

    Success measures

    • Retrieval precision at k
    • Trimming correctness on the permission test set

  3. 03Grounded generation

    Weeks 10–16

    Add the orchestration service and model gateway. Enforce citation-per-claim and the abstention threshold. Run the evaluation harness on every change from this point onward.

    Milestones

    • Citations enforced
    • Abstention threshold calibrated
    • Harness wired into the change process

    Success measures

    • Groundedness score
    • Citation correctness rate
    • Abstention rate

  4. 04Governance and red teaming

    Weeks 14–20, overlapping

    Adversarial testing for indirect prompt injection through ingested content, data leakage via crafted queries, and ACL edge cases involving nested groups. Findings gate the pilot.

    Milestones

    • Injection test suite executed
    • Leakage attempts documented and mitigated
    • Audit trail reviewed by compliance

    Success measures

    • Injection attempts blocked
    • ACL edge cases resolved

  5. 05Limited pilot

    Weeks 20–28

    One or two departments with a bounded corpus. Sampled human review of answers, and a visible feedback path so users can flag a wrong answer in one click.

    Milestones

    • Pilot cohort onboarded
    • Review cadence running
    • Feedback loop closing

    Success measures

    • Answers sampled and reviewed
    • Flagged-answer rate
    • Expert escalation volume

  6. 06Scale decision

    Quarter 2 onward

    Expand corpus and cohort only where evaluation supports it. Autonomy is revisited here as its own decision with its own approval, not as a continuation.

    Milestones

    • Scale criteria met or not met
    • Autonomy decision explicitly taken

    Success measures

    • Sustained groundedness at larger corpus size
    • Cost per answered question
12

Business Impact

These are the quantities the architecture is designed to move. The basis of each is stated so a reader can disagree with the reasoning rather than the number.

None of these is a measured result. This is a hypothetical study; no system was deployed and no production data exists. Estimated means derived from stated assumptions, Projected means a modelled forward view, Illustrative means the figure exists to show the shape of an argument.

Time to a cited answer

-40%

Projected

Modelled against the task-based baseline defined in Discovery, assuming retrieval precision targets are met. Untestable until that baseline is captured — which is the point of sequencing it first.

Groundedness on the evaluation set

≥ 90%

Estimated

A target derived from published RAG evaluation practice, not an observation. Set as a gate for scaling rather than a claim about performance.

Expert escalations for routine questions

-35%

Illustrative

Illustrative. Depends entirely on how much of the current expert load is routine and retrievable, which was not measured in this hypothetical scenario.

Unauthorized content in retrieved context

0

Projected

A design requirement rather than a forecast. Trimming happens before ranking, so the intended count is zero by construction — and is verified continuously against the permission test set rather than assumed.

Cost per answered question

Bounded

Illustrative

Deliberately not a number. It depends on model choice, context size and abstention rate, none of which are fixed at design time. Publishing a figure here would be false precision.

13

Risks & Constraints

Five risks, including the ones that argue against building this at all.

RiskCategoryImpactProbabilityMitigation
Hallucination — a fluent, confident answer unsupported by the retrieved contextAI hallucinationHighMediumConstrain generation to retrieved context, require a citation per claim, enforce an abstention threshold, and score groundedness continuously against a fixed question set. Accept a higher abstention rate as the price.
Indirect prompt injection through ingested content instructing the modelSecurityHighMediumTreat all retrieved content as untrusted data rather than instruction, separate system and context channels, red-team with injected documents before pilot, and keep the assistant read-only so a successful injection discloses rather than acts.
Data leakage through stale or incorrectly resolved ACLsPrivacyHighMediumIngest ACLs with content, re-index on permission change, resolve group membership at query time rather than caching it, and maintain a standing permission test set covering nested and inherited cases.
Model or vendor dependency — behaviour changes underneath a pinned promptVendor dependencyMediumHighAbstract the model behind a gateway, pin model versions explicitly, re-run the evaluation harness before any version change, and keep prompts and evaluation data portable.
Retrieval quality degrades silently as the corpus growsData qualityMediumHighTreat the evaluation harness as production monitoring rather than a build-time gate, alert on groundedness and abstention drift, and re-tune chunking when corpus characteristics change.
Adoption failure — users try it twice, get an abstention, and return to asking colleaguesOrganizationalMediumMediumFrame abstention as a routed question rather than a dead end, make the feedback path one click, and scope the pilot corpus tightly enough that early answers are good.
14

Security & Governance

Identity and authorization

Retrieval executes in the asking user's identity. The assistant holds no standing read access to the corpus, which means a defect in the assistant cannot become a tenant-wide disclosure. Authorization trimming is applied as a pre-ranking filter, because content that reaches the context window has already leaked regardless of what the response says.

Data governance

Corpus scope is an explicit governance decision with a named approver. Adding a content source is not a configuration change. The audit store holds source identifiers rather than document bodies, so it does not quietly become a second, less-governed copy of the corpus.

Prompt and model governance

Prompts are versioned artefacts subject to review, not strings edited in production. Model versions are pinned, and the evaluation harness runs before any change takes effect. Content filtering sits at the gateway, which is also the single egress path to the provider.

Auditability

Every interaction retains question, retrieved source identifiers, model version and response. This is what makes a post-incident question — "why did it say that" — answerable. Without it, the honest answer is that nobody knows.

Responsible AI

Users are told they are interacting with an AI system, answers carry citations, and abstention is visible rather than disguised. The system informs and does not decide, and that boundary is enforced architecturally rather than by policy alone. The NIST AI Risk Management Framework and the OWASP Top 10 for LLM Applications are used as the structure for risk review (see References).

15

Trade-offs

What is given up, and why it is acceptable here.

Groundedness vs Coverage

A strict abstention threshold means fewer questions answered. Accepted: a confident wrong answer is acted upon, an abstention is routed to a human. The failure costs are not symmetric.

Identity-aware retrieval vs Latency and simplicity

Resolving group membership and filtering per query costs milliseconds and real complexity. Non-negotiable — the alternative is a disclosure risk.

RAG vs The fluency of a fine-tuned model

A model tuned on the corpus may read more naturally. Rejected because it cannot cite, cannot be permission-trimmed, and cannot be updated when a document changes.

Read-only assistance vs End-to-end task completion

Declining tool access leaves multi-step savings on the table. Accepted deliberately: autonomy changes the risk class, and the observability to manage it does not yet exist.

Model portability vs Provider-specific capability

A gateway abstraction forgoes some vendor-specific features. Accepted to keep the most volatile component replaceable.

Auditability vs Storage cost and privacy surface

Retaining questions and sources creates its own governance obligation. Mitigated by storing identifiers rather than bodies, and by applying retention to the audit store like any other data.

16

Strategic Recommendation

A clear decision, with the conditions it depends on.

Build Option B — an identity-aware RAG assistant, read-only, with abstention as a first-class outcome — and keep Option A’s improved search available alongside it rather than replacing it.

Option A alone does not synthesize and leaves the expert bottleneck intact, but it remains the better answer for questions with a single authoritative document, and it costs little to keep. Option C raises the risk class rather than extending the capability: with tool access, a successful prompt injection acts instead of merely disclosing, and the observability required to detect a bad multi-step action does not yet exist in this scenario. Option B addresses the actual problem — synthesis with attribution — while keeping authorization enforced at retrieval, where it can still be enforced. The decisive argument is failure-mode asymmetry: RAG’s worst case is a wrong answer a human can check against its own citations; an agentic system’s worst case is a wrong action already taken.

Conditions

  • Authorization trimming is verified against a standing permission test set before any pilot, including nested and inherited group cases
  • The evaluation baseline exists before generation is built, or no quality claim afterwards is testable
  • Abstention is framed in the experience as a routed question, not as a failure
  • The assistant remains read-only until observability is demonstrably mature
  • A named owner accepts accountability for corpus scope decisions

Risks

  • Organizational pressure to add tool access informally, one connector at a time
  • Evaluation discipline decaying once the system is in production and attention moves on
  • Corpus scope expanding by request rather than by decision

Next steps

  • Agree the initial corpus scope and confirm ACL availability through Microsoft Graph
  • Build the evaluation question set with subject matter experts before writing retrieval code
  • Prototype retrieval with security trimming and measure it in isolation, without generation
  • Red-team indirect prompt injection with deliberately poisoned documents before the pilot
  • Set the abstention threshold from measured retrieval relevance, not from a target answer rate
18

References

Public vendor documentation and recognised public frameworks. No proprietary or client material is cited.

  1. Retrieval Augmented Generation (RAG) in Azure AI Search — Microsoft Learn · Documentation Source (opens in a new tab)

    Basis for the retrieve-then-generate pipeline.

  2. Security trimming for Azure AI Search results — Microsoft Learn · Documentation Source (opens in a new tab)

    Supports the decision to filter at retrieval on ingested ACLs.

  3. Microsoft Search and Graph connectors overview — Microsoft Learn · Documentation Source (opens in a new tab)

    Content and permission ingestion path.

  4. Content filtering in Azure OpenAI — Microsoft Learn · Documentation Source (opens in a new tab)

    Gateway-level content filtering referenced in Security & Governance.

  5. Microsoft 365 Copilot overview — Microsoft Learn · Documentation Source (opens in a new tab)

    Reference point for the build-versus-adopt comparison.

  6. AI Risk Management Framework — NIST · Standard Source (opens in a new tab)

    Structure used for the risk review.

  7. OWASP Top 10 for Large Language Model Applications — OWASP · Standard Source (opens in a new tab)

    Source for the prompt injection and data leakage threat model.

17

Key Takeaways

01

RAG is preferred over fine-tuning here not because it answers better, but because it can cite, can be permission-trimmed, and can be updated — three properties fine-tuning cannot offer at any quality level.

02

Authorization must be applied before ranking. Content that reaches the context window has already leaked, whatever the response says.

03

Traditional search is still the better answer for single-document questions, and keeping it costs almost nothing.

04

Agentic autonomy is a change of risk class, not a roadmap increment: read-only injection discloses, tool-enabled injection acts.

05

Abstention is a feature. A system that says "I don’t know" earns the trust that makes its answers usable.

06

Without an evaluation baseline captured before build, every later claim about answer quality is unfalsifiable.

07

The model is the most replaceable component in this design, and should be architected as such — the retrieval and permission layers are where the durable value and the durable risk both sit.

Working through a similar architecture or AI decision? I am happy to talk it through.

Get in touch