An agent that plans its own steps and calls your internal tools is a genuinely useful pattern and a genuinely risky one. The difference between the two is entirely in the boundaries you engineer around it, and those boundaries are what demos leave out.
Agents suit messy inputs, not messy outputs
The right place for an agent is a workflow where the input is unstructured and unpredictable — an email request, a support ticket, a document of unknown format — but the actions available are few and well defined. Rule-based automation breaks on inputs nobody anticipated; that is precisely the gap an agent fills. Where the input is already structured, a script is cheaper, faster and more reliable.
Use an agent where the request is unpredictable and the actions are not.
Tools are the security boundary
An agent can only do what its tools permit, so tool design is the whole safety story. Each tool should be a narrow, validated function with scoped credentials — not a general database client, not a shell. If a tool can only move a ticket between three states, a confused agent can at worst move a ticket to the wrong state, which is recoverable.
Shadow mode before autonomy
Run the agent for a period where it proposes actions without executing them, and compare its decisions against what your team actually did. This gives you a measured accuracy rate before anything is at stake, and it surfaces the categories where the agent is unreliable. Teams that skip this step discover the error rate through incidents instead.
Loops, ceilings and budgets
A confused agent does not fail loudly; it retries. Step ceilings, wall-clock timeouts and spend caps per task are mandatory, not defensive extras. Any agent capable of calling a tool that calls another tool needs a hard limit on total steps, because the failure mode is an expensive infinite loop discovered on the invoice.
The audit trail is the product
When an agent does something surprising — and it will — you need to reconstruct exactly what it saw, what it decided and why. Log the reasoning trace, the tool calls, the arguments and the results, in a form a non-engineer can read. Without this, every incident becomes unexplainable, and unexplainable systems lose the trust they need to keep operating.
Where we have seen them pay
The consistent wins are unglamorous: triaging and enriching inbound requests, reconciling records across systems that disagree, gathering research from several sources into a structured summary, and multi-step back-office processes that currently consume hours of clicking. Each removes a repetitive middle and routes the judgement to a person.
