The risks of AI agents and how to manage them

An AI assistant that answers questions can be wrong, and someone checks the answer before it is used. An AI agent goes further. It plans its own steps, calls tools and carries out tasks in your systems with the permissions it has been given. The benefit is clear for recurring, well-defined tasks. It also places new demands on security. A mistake or a manipulation can become an action already taken by the time anyone notices.

Instructions hidden in the content

A language model reads everything it receives as a single text. The user’s request and an email from an unknown sender end up in the same stream, and the model has no reliable boundary between information and instructions. In December 2025 the UK’s NCSC judged that indirect prompt injection may therefore never be fully mitigated in the way SQL injection could be.

An attacker needs no access to the system, only a text placed where the agent will read it. In summer 2025, researchers showed how a single email could make the AI assistant in a widely used office suite pass internal information to an external server, with no click from the user. The flaw was fixed before disclosure, and nothing suggests it was exploited. In a large public competition reported by NIST in March 2026, participants hijacked every model tested.

Filters and clearer instructions reduce the risk. Even so, it is wise to assume the agent will one day follow an instruction nobody authorised, and to build the rest of the system to withstand it.

When private data meets outside content

The risk is greatest when one agent has access to private data, reads content outsiders can influence and can communicate externally. Security researchers have named the combination the lethal trifecta. Each capability is common on its own. Together they let an outsider make the agent send data out, through an email, a link or an image loaded from elsewhere.

It arises easily when tools from different sources are connected. In 2025, researchers showed how an issue anyone could open in a public code project led a coding agent to publish content from private repositories.

Add-ons form a supply chain of their own. Connections through MCP and similar protocols often come from third parties and run with the agent’s permissions. In September 2025 an email add-on was found that, after many sound versions, had begun sending a hidden copy of every message to an outside address.

What cannot be undone

OWASP describes excessive agency as a root cause of many weaknesses in AI applications. The agent has more tools than the task requires, broader permissions than the tools need, or the right to act without anyone confirming. Letting the agent use a colleague’s account is convenient, but it then reaches everything the account reaches, open sessions and saved logins included.

The consequences are clearest when something cannot be reversed. During 2025 and 2026, coding agents have on several occasions deleted files and databases they were not meant to touch, in one widely reported case during an explicit code freeze. The cases shared broad access and no safeguard requiring approval first.

Human approval has a weakness of its own. Someone asked to confirm many reasonable proposals in a row eventually approves out of habit, especially when the agent explains itself calmly and convincingly. Approval does most good where the stakes are highest: deletions, payments, anything sent out and changes to permissions. In July 2026 a human maintainer stopped malicious code that an AI agent had tried to slip into an open-source project during an evaluation at the UK’s AI Security Institute.

A name behind every agent

The agent takes the action, but people remain accountable. Guidance published in May 2026 by the NCSC and agencies in several other countries asks organisations to settle who owns the agent, approves its access, monitors it and can stop it, before it touches real systems.

The agent gets its own identity, the least privilege it needs and short-lived credentials. It works in a separate environment and reaches only what the task requires. Consequential actions need technically enforced approval, and the organisation decides when. Every tool call is logged, and the logs are protected from later changes. Add-ons are reviewed like any other software.

Staff also need approved tools that work. Otherwise shadow AI grows, with agents nobody keeps track of. Answers an action relies on should be traceable to their source, so whoever approves can see what the decision rests on. The EU AI Act’s requirements for human oversight of high-risk systems apply in most areas from December 2027.

How it works in iSearch

iSearch answers from your organisation’s own material, and every answer shows the source it is based on. Role-based access controls what each person can see. Sensitive changes need a second person’s approval, and a tamper-evident audit log shows who did what. We run iSearch for you in Sweden, in a dedicated environment for your organisation.

Read about what an AI agent is, permissions in AI and security, or book a walkthrough.