Case study · AI systems engineering

Giving an AI assistant real access to a company's data

Backend and integration work on an internal AI assistant.

Go
Python
Vue 3
MongoDB
Valkey
OAuth 2.0 + PKCE
Kubernetes
AWS

An internal AI assistant is only as useful as what it can reach. Ours could reason well and knew nothing. It had no access to the documents, threads, calendars or mail the questions were actually about. I built that access. What I learned is that connecting a language model to a company's real data is mostly not an integration problem. The APIs are documented and the OAuth flow is standard. The problem is that the moment an assistant can act, every mistake it makes becomes someone else's inbox. Almost all of the engineering went there.

437plaintext tool-result records encrypted in place across 64 conversations, 0 failures, verified three ways

The problem

People wanted to ask the assistant ordinary things. What’s in this policy document, is this person free this afternoon, what did they say about that in chat, and then have it do something about the answer. Straightforward as a feature request. Four constraints made it hard, and they turned out to be the real design inputs.

It must never see more than the person asking

A service account with organisation-wide read access would have been far simpler to build, and would have quietly handed every user a data-access upgrade.

The model must never see the credential

Anything that enters the model's context can leave it again, in a reply, a log, or a stored conversation.

A wrong action is not like a wrong answer

A bad summary is a nuisance you correct. A message sent to the wrong person cannot be recalled.

Content pulled in doesn't disappear afterwards

It lands in conversation history, audit records and caches, each with its own retention and access rules.

None of these are about the API surface. All four shaped the architecture.

What I built

Corporate identity-provider OAuth, added to an existing chat platform, plus four tool servers that use it.

Already existed

Generic OAuth 2.0 flow
PKCE support
Encrypted token storage
Background token refresh
Per-user tool connections

What I added

The provider as a first-class auth type across the nine files that branch on auth type
A server-template merge, so configured credentials were actually present at token-exchange time
Public-client configuration, meaning code exchange with no client secret at all

Admin setup reduced from six fields to one.6 ──▸ 1
Endpoints and scopes are derived server-side from a single client identifier, so they can’t be mistyped, or tampered with per user.
Four tool servers behind one consented identity.
Documents, chat, calendar and mail. A user authenticates once; every server uses that same delegated token.
Delegated, read-heavy permissions, with send-mail deliberately never requested.
The assistant drafts into your drafts folder. You press send. That removes a class of abuse rather than rate-limiting it.
Per-server guardrails.
Send rate limits, duplicate suppression, bounded reads so one question can’t pull thousands of private messages into context, and a block on messaging anyone outside the company.
A human-in-the-loop confirmation gatein front of every state-changing action.
Encryption of tool-result content at rest,plus a migration of the records that already existed.

Architecture

Two flows, and they share almost nothing. Conflating them is the fastest way to misunderstand where the security properties come from.

Consent: once per user, per server

The redirect lands on the frontend, which then posts the code to Go for the exchange. That split is why the app registration had to be a public client rather than a web or single-page app. One of those rejects a secret-less exchange, the other refuses a server-side one. Working that out took longer than the code did.

Every subsequent tool call

Tool call: the credential path stays outside the model

The tool servers are independent microservices, each on its own deployment pipeline. Auth is delegated per user rather than a service account, so the assistant inherits the requesting person’s own permissions and cannot exceed them. That is constraint 01 enforced by the identity provider, not by code I wrote and could get wrong.

The token makes exactly two hops: backend to tool server, then tool server to the vendor API. It is the same token both times, it travels as a header both times, and at no point is it a value the model can read or write. One consented identity is shared across all four servers, so adding a fifth is a registration change rather than another consent screen for every user.

Four problems worth describing

The credential that went to the wrong place

Symptom: A tool failed with an empty-token error while the logs cheerfully reported that the token had been supplied.
Diagnosis: The backend decided which tool parameter should receive the OAuth token by scanning each parameter’s description for substrings like token, key, auth. One tool took a person’s display name, and its description read “if omitted, returns the authenticated user”. Authenticated contains auth. So the matcher flagged name as a credential field and the backend wrote an access token into it. The token wasn’t missing. It had been posted into a search box.
Decision: The obvious fix was a better pattern list. Looking for where the injection happened, I found something better: the backend was already attaching the token as an HTTP Authorization header on the connection to each tool server. That path existed and worked. The parameter injection was a second, redundant mechanism, and the only one that could ever put a credential somewhere the model could see it. So I deleted it. Each server now reads the token from its incoming request context, via one shared helper that replaced four near-identical copies.
Result: No tool declares a token parameter, so the model is never offered one, and no heuristic has to guess correctly.

A guessing mechanism whose failure mode is credential misplacement shouldn’t be made more accurate. It should stop existing.

The confirmation gate, and why prompt injection isn't patchable

Symptom

Colleagues testing the assistant found they could plant instructions in content it read, in a chat message or a document, and get it to act on them. Reply to someone. Book something. The content the agent consumes is untrusted, and the agent cannot tell data from instruction.

Diagnosis

This one is worth being honest about: it is not solved at the model layer, by anyone. A system-prompt instruction saying don’t act on behalf of others is a stopgap rather than a control, and a sufficiently well-worded injection walks straight through it. Building an elaborate defence inside the tool layer would have been false security.

Decision

Move the decision to a human instead of trying to make the model incorruptible. Every state-changing tool now pauses: the proposed action is written to the cache with a TTL, a confirmation_required event streams to the UI, the backend blocks on a poll for up to sixty seconds, and a person clicks. Injected content can propose an action. It cannot click the button.

send_messagechat tool server · delegated

15s

To: A colleague (internal recipient)
Message: Thanks, I've reviewed the draft policy and left two comments on the retention section. Happy to walk through them tomorrow.

Reconstruction of the shipped UI. Countdown compressed to 15s for the demo; the real gate blocks for 60.

Two bugs

The model retried after a denial. Refusing a send produced two more confirmation prompts. The tempting fix was a sterner instruction in the error message. The real fix was an explicit rejectedAction flag gating the recursive tool loop, so the retry is structurally impossible rather than discouraged.

On the deployed environment, the confirmation card never appeared. It worked locally. The reverse proxy was buffering the confirmation_required event for the entire sixty-second block, then releasing it in the same batch as the timeout, so the UI set the pending state and cleared it in the same tick, before a frame could render. Fixed with a keepalive goroutine emitting a named event every two seconds, which forces the buffer to flush.

The buffering one is the bug I’m most pleased with. It presents as a frontend rendering fault, isn’t one, and only resolves if you reason about what sits between the two.

On what the gate costs, since a control that adds a wait should say so: the machine-side cost is bounded by the poll interval. The backend checks the cache twice a second, so a click is noticed within about half a second, and the sixty seconds is a ceiling on waiting for a person rather than a delay. Everything else is human decision time, which I never measured and won’t estimate. The honest version is that the gate does not add latency so much as insert a person, and the wait is however long they take to read one sentence.

Finding a data-protection problem in my own feature

Symptom: Nothing was failing. During a security review I went looking for where tool output came to rest, and found that my new tools had changed what that question meant. The platform’s earlier tools returned things like the current date. Mine returned private conversation histories and document text. Conversation records were encrypted at rest; the tool-result copies of the same content were not.
Diagnosis: Not a defect anyone had introduced. It was a design that had been proportionate to the old tools and stopped being proportionate the moment mine shipped. Plaintext content sat in two places: an audit collection, and the conversation record the model reads back as history.
Decision: The audit collection was write-only. Nothing read it, and the same content already lived in the conversation record. Redundant plaintext is pure liability, so it was deleted rather than encrypted. The copy that is load-bearing was encrypted in place. Two details mattered. The field was an any type holding mixed shapes, so it’s JSON-marshalled before encryption and unmarshalled after to preserve fidelity for the model. And I added a dedicated per-field flag rather than reusing the document-level one, because reusing it would have made the new decrypt path attempt to decrypt existing plaintext and destroy the assistant’s memory of every past tool call.
Migration: 437 existing records across 64 conversations. Backed up outside the repository first, run idempotently on startup, and verified three ways: the encrypted count flipped, the inverse query for remaining plaintext returned zero, and, the one that actually mattered, a follow-up message in a migrated conversation still produced a coherent answer. Nothing was proven by the fact that it compiled.

437records migrated

64conversations

0failures

0plaintext remaining

Two scanner findings, and the one that came back

Symptom: An automated security review on the pull request flagged two input-validation gaps. SSRF: provider URLs came from user-supplied configuration and were used directly as the target of server-side requests, reachable at cloud metadata endpoints and internal cluster addresses. Header injection: user-supplied values became HTTP header names and values unsanitised, and basic credentials were concatenated rather than encoded.
Decision: A central validator for each, called at both configuration-save time and request time for defence in depth: scheme enforcement, a provider hostname allowlist, and rejection of private, loopback and link-local resolutions; strict header-name patterns with a denylist for the headers that can rewrite a request, control-character rejection on values, and proper base64 encoding for basic auth. Thirty unit tests across the two fixes, each asserting a specific rejection.
The follow-up: The reviewer came back on the fix, correctly: validating the resolved IP before the request leaves a time-of-check-to-time-of-use window, because a hostname can resolve differently between the check and the connection. Closed by moving the check into a custom dialer that re-resolves and re-validates the destination at TCP connect time, keeping the earlier validation as cheap early rejection.

The interesting part isn’t the first fix. It’s that a fix which looked complete wasn’t, and the right response to being told so was to go a layer down rather than argue the window was too narrow to matter.

30unit tests

2findings cleared

1follow-up caught by review

Adjacent: the cost of having tools at all

Every tool a model can call has to be described to it on every single request. With four new servers and a few dozen tools, those schemas became a large fixed token cost paid on every message, including the ones that needed no tools whatsoever.

The fix ranks tools by embedding similarity against the user’s message and passes only the top K. Note where this sits. It happens before the model call, and has nothing to do with the auth path above:

Tool selection: before the model call, and not part of the auth path

Three things about it matter more than the technique, which is unremarkable:

It reused infrastructure rather than adding it. The platform already generated embeddings; this needed a similarity function and a cache, not a new service.

The invariants were written down before the code. Access control is never widened by selection; any failure falls open to the full tool set rather than an empty one; a user’s own preferences win. Selection is an optimisation, and an optimisation that can deny access or silently drop capability isn’t one.

Measurement killed part of my own design. I had built an intent-detection layer to force certain tools into the set. Instrumented in real use, its threshold never once fired, because the similarity ranking was already surfacing those tools. I deleted it and reduced K. The measured token reduction on tool-heavy requests ranged from roughly a third to over ninety per cent depending on the message.

Where it landed

Verified

Four tool servers live behind a single consented identity
The credential is never model-visible. No tool declares a token parameter
Confirmation gate working on a deployed environment, not just locally
PII exposure closed and proven closed. 437 records migrated, zero failures, zero plaintext remaining
Both scanner findings cleared, including the TOCTOU follow-up
Measured token reduction on tool-heavy requests
Tested end to end by several colleagues in a group session, which is where the injection findings came from

What this doesn't solve

Prompt injection has no clean fix at the model layer, not here and not in the field generally. The gate is a blast-radius control: injected content can still get the assistant to propose an action, and the design assumes that rather than pretending otherwise
Discoverability amplification is the real residual risk, and it isn't a code issue. The assistant can't exceed anyone's permissions, but it does make an over-shared document trivially findable where previously it was technically accessible and practically buried. That's pre-existing data hygiene the tooling surfaces, and it's the point I flagged hardest when handing over

What I'd do differently

Design the credential path as header-only from the start. I arrived at the right architecture through a bug. The parameter-injection mechanism should never have been the thing I inherited and worked around; it should have been the first thing I questioned.

Treat “what does this tool return, and where does that get stored” as a design-time question. I found the plaintext problem in review. It was answerable on day one, from the tool’s return type alone.

Route every tool error through redaction from the beginning. Adding it to one server and intending to backfill the rest is how you end up with a security control that exists in the codebase and not in the system.

Put the rate limits in shared state on the first pass. I built them in memory, which makes them per-container rather than per-user: correct for one instance, wrong the moment it scales. Cache-backed from the start would have cost an afternoon.