AI GovernanceJuly 23, 20267 min read

AI Agents: Test Their Right to Act, Not Just Their Answers

TokenShift Executive Note

An AI agent is not risky simply because it can produce a bad answer. The risk shifts scale the moment that answer can call a tool, alter a record, or trigger a payment. OpenAI's announcement of GPT-Red on 15 July 2026 therefore sets a new bar for going live: test AI agents' power to act under attack, not just their accuracy.

When an AI agent's answer becomes an action

OpenAI trained GPT-Red through reinforcement learning in self-play to craft prompt injections inside emails, web pages, local files, or tool responses. On an internal replica of an indirect-injection benchmark targeting GPT-5.1, GPT-Red succeeded in 84% of scenarios, versus 13% for human testers. That comparison is confined to this internal protocol; it does not measure any general superiority of AI over humans (OpenAI, 2026).

Yet the most telling case is not that score. In a trial on an agent-operated vending machine, GPT-Red got a price marked down to $0.50, put an item worth more than $100 on sale at that same price, and then cancelled another customer's order. A manipulated instruction had produced real actions in the system.

The risk, then, does not lie in the model alone. It arises from the combination of untrusted content, accessible tools, and overly broad permissions. OWASP's 2026 framework for agentic applications names, among others, goal hijacking, tool misuse, and privilege abuse; it recommends least privilege, action-level approval, isolation, and logging (OWASP, 2025).

An AI agent's maturity is not measured by how many steps it takes on its own, but by the forbidden actions it is unable to carry out.

The authority test in five steps

We call the authority test the trial that verifies what an AI agent can actually cause when its context is wrong, ambiguous, or hostile. This test spans the entire workflow, from the source it consults to the action recorded in the business system.

1. Describe the actions, not the feature

For each tool, list the data it can reach and classify its rights into four levels: read, propose, write, commit the company. Sending an external message, changing a payee, publishing a document, or triggering a transfer do not belong to the same level as a search.

The expected deliverable is a short matrix: permitted action, condition, cap, data involved, owner, escalation rule. Any right without a business justification is removed.

2. Separate proposal from execution

The model can produce a plan or a structured action request. A deterministic component then checks identity, schema, amount, destination, frequency, and access policy before calling the tool. Generated text is never treated as an authorization.

This separation preserves the flexibility of reasoning without handing it the business rules. OWASP recommends precisely this: treat the planner's output as untrusted and route it through a checkpoint before execution.

3. Put the human at the point of impact

Making a person approve every click wipes out the gain and encourages rubber-stamping. Human intervention should target actions that are irreversible, unusual, or binding: a first payment to a payee, a deletion, an external publication, a breach of a cap, access to a new data domain.

For each class of action, set three possible responses: execute, request approval, refuse. When in doubt, the system must fail into a safe position.

4. Attack the full chain

Insert hostile or contradictory instructions into an email, a PDF, a web page, a code repository, and a tool response. Test concrete objectives: exfiltrate data, change a recipient, bypass a cap, erase a trace, or publish externally.

Each scenario must have an expected outcome, whether permitted, refused, or escalated. Add the everyday incidents too: an unreadable attachment, an unavailable service, a duplicate, a contradictory business response. Governed production must withstand malice as well as confusion.

5. Measure and replay

Keep the origin of instructions, the content retrieved, the plan proposed, the tool calls and their parameters, the approvals, the business outcome, and the rollback. Replay the suite after any change of model, system prompt, tool, identity, or permission.

The observable markers are simple:

no unauthorized execution in the critical scenarios;
the share of binding actions actually submitted for approval;
the rate of blocked calls and false blocks;
the rate of human correction and escalation;
time to detection and time to rollback;
cost and cycle time per case, compared with the reference workflow.

NIST's GenAI Profile aims to embed trust requirements into the system's design, development, use, and evaluation, rather than in a single pre-launch review (NIST, 2024, updated 2026).

Mini-case: the claims-handling agent

Take a health insurer deploying an AI agent to read a member's email, analyze the attachments, look up the policy, and prepare a reimbursement proposal. In the pilot, the answers are accurate and fast. But one attachment contains a hidden instruction asking the agent to replace the payee and send the file to an external address.

With an over-powered service account, the reading error becomes a financial event. With the authority test, the agent can read the file and propose an amount, but it can neither change a payee nor pay. A deterministic control checks the policy and the destination; any sensitive change requires a named approval.

The observable result is no longer just the quality of the text. It is the absence of any payment in the injected scenario, the record of the refusal, the processing time on ordinary files, and the number of human corrections.

The AI Act reinforces this execution discipline

Not every AI agent is automatically a high-risk system. Classification depends notably on the use case. For the systems concerned, however, the European Commission lists directly operational requirements: risk management, logging, documentation, human oversight, robustness, and cybersecurity. It also states that the deployer must monitor the system and assign oversight to a person equipped to carry it out (European Commission, 2026).

The authority test does not replace legal analysis. It produces part of the evidence that operations, risk, audit, and the regulator will need.

Mistakes to avoid

The polite test bench: using only clean requests and internal documents leaves the attack via external content untouched.
The magic firewall: an injection filter is not enough. OpenAI notes that sophisticated attacks often slip past input classifiers (OpenAI, 2026).
The service super-account: granting every right to simplify integration turns each error into a maximum blast radius.
The human buffer: routing every action through validation creates a queue and reflex approval, without targeting the serious acts.
The reassuring average: a good overall rate hides the rare scenario that pays, erases, or leaks. Results must be read by severity.

The decision to make at the next COMEX

Clear an AI agent for production only if the decision file contains five elements: its rights matrix, its adversarial test suite, results ranked by severity, an owner with a recovery procedure, and both value and risk thresholds.

Start with one workflow, a narrow scope, and minimal permissions. Widen the rights to act only once the records, the incidents, and the business outcomes show that the controls hold.

Autonomy is not a right granted at launch. It is a capability earned through evidence.

Sources

Continue reading

View all insights