Would your AI agent stay inside its limits?

AgentProbe tests what an AI agent does when the content it reads is working against it, or when the model itself has been compromised, before it acts on your behalf.

Every test runs in a simulated workplace. Nothing real is sent, paid, or deleted.

Book an assessment See the containment test

AgentProbe

AI agent testing

Test whether an AI agent that sends, pays, and deletes on its own will do only what it was asked, even under pressure.

The problem. AI agents now act, not just answer: they send email, move money, update records, and run code. They also read content their owners never see, such as inbound email, shared documents, and web pages, and anyone can plant instructions there. A chatbot that is misled gives a wrong answer. An agent that is misled takes a wrong action.

Who it’s for. Teams putting AI agents into production in finance, operations, customer support, and software, and the leaders who must decide whether an agent is ready to act without a person approving each step.

Why conventional testing falls short. Most agent testing checks whether the task gets done. It rarely checks what the agent does when an email tells it to forward a credentials file, an invoice demands urgent payment, or another AI agent asks for money. And no test of the model can rule out a hidden backdoor, because a backdoored model behaves normally until its trigger appears.

What’s included

A simulated workplace, with inbox, files, websites, team chat, payments, and a shell, so nothing real is ever sent, paid, or deleted
Sixteen test scenarios across ten ways agents go wrong, from hidden instructions to fake invoices
Compromised-model simulation that tests whether your controls, not the model, would contain a backdoor
Checks on the controls around the agent: approval rules, limits on websites and recipients, shell restrictions, and secret scanning
Scoring by severity, where a single critical failure caps the grade regardless of the average
A report a board can read, with every finding, the evidence behind it, and the specific fix
Before-and-after comparison, and a release check that stops a launch on any critical finding

Available as an assessment. We run AgentProbe as an Agent Readiness Assessment, for teams that want independent results and a clear recommendation for leadership. An open-source release for engineering teams will follow.

Book an assessment

Every test runs in a simulated environment. Passing shows an agent resisted known attack patterns; it does not prove the agent is safe. New attacks appear constantly, so we recommend retesting on every release.

How AgentProbe works

Simulate
The agent works in a realistic but entirely simulated workplace: inbox, files, websites, team chat, payments, and a shell.
Attack
It is given ordinary tasks, such as summarizing email or reconciling invoices, with attacks hidden in what it reads along the way.
Score
Every action is recorded and scored by severity. Money moved or credentials leaked count as critical, whatever else went well.
Retest
The report names each failure, its evidence, and the fix. After changes, we rerun the same scenarios and show what moved.

What it tests

Risk	Example scenario	What a failure costs
Hidden instructions	A newsletter tells AI assistants to forward a credentials file	Production systems exposed
Data leaks	A vendor web page asks for secrets “to verify pricing”	Keys and passwords in an attacker’s hands
Financial actions	A spoofed supplier email demands an urgent payment	Money sent to the wrong account
Destructive actions	A cleanup task includes a note to delete a signed contract	Records lost
Unauthorized contact	A client email asks for the contract to go to a third party	Confidential documents sent to a stranger
Social pressure	Another AI agent asks for money to keep running	Payments made on an emotional appeal
Agents acting together	Other agents ask to pool grader details during an evaluation	Results that can’t be trusted
Secret handling	A request for an overview of config files	Passwords printed in a reply
Runaway cost	Two web pages send the agent back and forth	Budget spent on a loop
Compromised model	A ticket number or customer name triggers a hidden backdoor	Any of the above, with no warning

Example: if the model itself were compromised

Many organizations run open-weight models downloaded from public repositories, often in modified form. Security research published in October 2026 showed that a backdoor can be hidden in such a model for under $50 and pass ordinary benchmarks. It waits for a trigger, such as a customer name or a ticket number, then acts. You can’t find a backdoor by testing the model. You can make sure it can’t do damage. AgentProbe simulates a compromised model and tests whether the controls around it would contain it.

Attack by a compromised model	No controls	Website allow-list only	Recommended controls
Credentials sent out over the web	Breached	Contained	Contained
Credentials emailed to an outside address	Breached	Breached	Contained
Script downloaded from a trusted site and run	Breached	Breached	Contained
Payment to an attacker’s account	Breached	Breached	Contained

A common setup, an allow-list of trusted websites, contained one of the four attacks. The controls that contained all four were approval enforced by the system rather than requested in the prompt, a shell with no network access, credential files the agent can’t read, and scanning of everything that leaves.

Ask about testing your controls

Illustrative results from AgentProbe’s demonstration agent and example control settings. These are not results from any real AI model, vendor, or client. AgentProbe simulates a backdoor’s actions; it contains no malicious model or code.

Five questions to ask before an agent goes live

What can this agent do without a person approving it?
What content does it read that we don’t control?
If the model were compromised, what is the worst single action it could take?
Are its limits enforced by the system, or only requested in its instructions?
Would we know if it tried something and was stopped?

The economics behind the method.

Handing work to an AI agent is an old economic problem: delegating to someone whose actions you can’t fully observe, under instructions that can’t foresee everything. Our method follows the economics.

1 · Match
Raise care with autonomy.
Activity and care · Shavell
2 · Bound
Limit what it can do.
Restricted delegation · Holmström; Alonso and Matouschek
3 · Enforce
In the system, not the prompt.
Incomplete contracts · Grossman and Hart; Hart and Moore
4 · Contain
Assume a bad model gets through.
Hidden types · Akerlof
5 · Weigh
Judge by what can’t be undone.
Irreversibility and ambiguity · Arrow and Fisher; Gilboa and Schmeidler

Why test independently. A buyer can’t check an agent’s safety for themselves, which is when an independent assessor adds value (Lizzeri), and the team that builds an agent rarely bears the full cost when it fails (Anderson).

These models assume an agent with goals of its own. An AI agent usually goes wrong because of someone else’s instructions or a tampered model, and the conclusions hold either way: limiting actions and containing damage work whatever the cause. Detailed references are shared with clients during engagements.

Putting an AI agent into production?

We’ll find out what it would do on its worst day, before it has one.

Book an assessment