AI security and governance

Security is being asked to sign off on something with no precedent.

We help security and risk teams work out what an agent estate can actually do, what would happen if one misbehaved, and what evidence they need before anyone puts their name to a production rollout.

The situation

The threat model you have was written for software.

Traditional application security assumes the thing you are defending follows its instructions. An agent takes instructions from whatever text reaches it, holds credentials, calls other systems, and can be persuaded. The controls that worked for deterministic software do not map cleanly onto that.

So security teams end up in an uncomfortable position: asked to approve something quickly, without an established threat model, a shared vocabulary for the risks, or a way to evidence what happened afterwards. Blocking is unpopular. Approving without those things is worse.

What we work on

Making agent risk something you can reason about.

The work is usually a mix of threat modelling, control design and getting the evidence story straight before a review rather than during one.

Agent threat modelling

Working through what your agents can reach, what an attacker would do with that access, and which of your existing controls still apply. Grounded in your actual estate rather than a generic checklist.

Prompt injection and tool misuse

Where untrusted content can reach an agent, what it could make that agent do, and which mitigations are worth the cost. Includes the indirect paths through documents, tickets and retrieved content that teams tend to miss.

Least privilege for non-human callers

Most agents hold far more access than their job needs, because the credential to hand was the broad one. We work through what each agent genuinely requires and how to get from here to there without breaking things.

Auditability and evidence design

Deciding what has to be recorded, in what form, and where it lands, so that a question six months later has an answer. This is the part that usually gets designed last and should be designed early.

Human oversight that works

Which actions stop for a person, who that person is, and how they are given enough context to make a real decision rather than rubber-stamping a queue.

Getting through the review

Preparing the material your own risk function, auditors or regulators will ask for, and rehearsing the questions before they are asked in a meeting that matters.

How it runs

Threat model first, controls second, evidence third.

Doing these in the wrong order is the common mistake. Controls chosen before the threat model tend to be the ones a vendor was selling.

01

Map what exists and what it can reach

The agents running, the systems they touch, the credentials they hold, and who owns each one. Frequently this stage alone changes the conversation, because the estate is larger than the register suggested.

02

Model the threats that apply

What could go wrong given that specific estate, ordered by plausibility and blast radius rather than by how alarming it sounds. We are explicit about the risks we think are overstated as well as the ones that are not.

03

Design the controls and the evidence

What to enforce, where to enforce it, and what record it leaves behind. Delivered as something your engineering team can implement and your risk function can accept.

What you end up with

Something your risk function will actually accept.

  • A threat model written against your estate, not a generic one
  • An inventory of agents, the systems they reach and the access they hold
  • A control design covering identity, policy, approval and containment
  • An evidence and logging design mapped to the questions you will be asked
  • A prioritised remediation plan with effort and impact against each item
  • The risks you are choosing to accept, written down rather than implied
Common questions

Questions we get asked about this work.

Do you do penetration testing of AI systems?

This engagement is design and assurance work rather than offensive testing. We model the threats, design the controls and prepare the evidence. Where hands-on testing is the right next step we will say so, and scope it separately rather than folding it into an advisory piece.

How is this different from our existing application security process?

Most of your process still applies and we are not going to duplicate it. The additions are the parts that assume a deterministic system: an agent that takes instructions from untrusted content, holds credentials, and delegates to other agents needs a threat model that accounts for all three.

Our agents are built by several different teams. Does that matter?

It usually matters more than the frameworks involved. Estates built independently end up with inconsistent identity, logging and limits, so the security question becomes a coordination question. We tend to start with what has to be standard versus what can stay per team.

Does this end with you selling us the platform?

No. The output is a threat model, a control design and an evidence plan, which are useful whatever you implement them with. If KPATH AMP is a sensible way to deliver some of it we will say so, and equally we will tell you when your estate does not warrant a control plane yet.

Related: the KPATH AMP platform, shadow AI agent discovery, what the EU AI Act asks of agent estates.

Get ahead of the security review, rather than through it.

A free 30 minute conversation about your agent estate and what your risk function is likely to ask.