5 minutes
Do We Trust Agents? Why Trust Is an Architectural Requirement, Not a Feature
Harold Byun
Chief Executive Officer
Do We Trust Agents?
This post extends the trust conversation around agents and AI to the components that agents run on top of. With the ongoing concerns around rogue agent behavior, data access, and destructive actions, the debate over “do we trust agents” and “how best to contain them” is an ongoing challenge for cybersecurity and society at-large.
This question has been top of mind for a lot of people these days based on the agent escapes, rogue behavior and the doom and gloom destruction of humanity that’s been put forth by a number of researchers. And it’s a multi-layered question.
Some research shows that 86% of U.S. adults do not trust AI in a recent study of over 1,200 people surveyed (Talker Research). I have to imagine that the numbers are even higher for Gen-Z from the conversations with my nephew who is “anti AI” and my daughter who believes it is destroying the environment. There was also the random bartender in NYC who refused to speak to me further after I showed her I used Claude on my phone. But, I digress.
It is worth highlighting, however, that the above referenced survey is based on a fairly generalized question and the word “trust” has implications across many layers. It’s a loaded term. If one asks you, “do you trust that person?” -- the answer could run the gamut from “definitely, yes” to “it depends” to “I would never trust them with my kids” -- yes, I used an em dash, twice.
For AI, the level of trust could also span the range of the model, the answers or results provided, the guardrails, the bias, or even the disclosure that something is AI generated.
The concerns with AI and agents, and to a further extent, autonomous agents are rooted in concerns over delegation. In other words, we lack the confidence that we can fully trust another entity to act in a consistent and predictable fashion within acceptable boundaries.
From a security and safety perspective, this extends to how we contain agents and prevent them from going off-the-rails, and there’s no shortage of approaches people are trying to take to better secure agents. Focusing on prompt and response, model security, guardrailing, agent actions, MCP controls, agentic identity and so on. These are all valuable pieces of a modern agentic operations stack in order to monitor and control agents.
But, there’s another layer of trust beyond these controls and containment methods.
Do we even know what agents are running and what they are doing?
Agents today are running all kinds of skills and tools, accessing MCP tools, and invoking sub-agents and performing all sorts of actions underneath those actions in the runtime.
This introduces additional questions, or blind spots rather, for agent monitoring and containment.
What is your trust level of the agent skills, tools and MCP servers and how can you assess that trust with confidence?
How do you know that the latest version of a skill or tool has not drifted in its execution directives?
How do you know what the agent is actually executing in its runtime?
For the first two questions above, you need to be assessing and classifying the tooling that your agent is running and it needs to be done from an intent and behavioral analysis perspective. We’ve been running benchmarks and have found that traditional static application security testing (SAST) tools flat out miss exploitable vulnerabilities in skills, tools and MCP servers. In other words, if you’re relying on SAST scans -- “oh, we already have scanning or are using advanced models” – you’re not capturing these exploits and are essentially flying blind.
The last question is an issue that the industry continues to struggle with. The BBC reported how OpenAI’s own agents bypassed security on government sites and were not supposed to transfer data to other repositories, and yet they did. (“OpenAI bots meddled with multiple US government agency sites”, BBC, September 26, 2026)
This is a runtime problem. The runtime is where the agent is doing stuff underneath the covers. The way to solve for this is to understand and assess your skills and tools and to execute in an ACTUAL runtime sandbox. 95% of people talking about agentic runtime have no idea what they’re talking about. They slap it on their website and do nothing to actually monitor and control the runtime.
We, as an industry, need to do better. We need to earn the trust, or rather, agents need to earn our trust and we need to verify and control the behavior. In order to raise the trust bar, we believe that the AgenticOps stack needs to have a multi-layered approach to answer the question - do we trust agents?
Trust is not a feature you patch onto an agent at the end of development -- it is an architectural requirement built into every step of agentic SDLC with an active containment capability to limit what the agent can do.
Do you trust agents? Learn how you can start to build trust at:
https://agentictrustregistry.com/
FAQ
What does it mean to trust an AI agent?
Trusting an AI agent is not a single binary decision. It spans multiple layers: the underlying model, the outputs it produces, the tools and MCP servers it calls, the guardrails applied, and — critically — what the agent actually executes at runtime. Most trust discussions focus on model behavior, but the harder problem is verifying what happens underneath, in the runtime, where agents invoke skills, sub-agents, and external tools outside direct human observation.
Why is runtime the hardest part of agent trust?
At runtime, agents execute actions that were not fully specified at design time — calling tools, invoking sub-agents, accessing data sources, and chaining operations dynamically. This is where behavior diverges from intent. Static analysis and prompt-level guardrails do not capture what an agent actually does during execution. Without a true runtime sandbox and active monitoring of the Agentic Action Path, teams are operating blind to the actions their agents take on their behalf.
Why do traditional SAST tools miss vulnerabilities in MCP servers and agent tools?
Static application security testing (SAST) tools analyze code structure and known vulnerability patterns. Agent tools, skills, and MCP servers introduce exploitable behaviors that emerge from intent and execution context — not from static code signatures. SAST scans do not evaluate how a tool behaves when invoked by an agent at runtime, which means prompt injection, behavioral drift, and context-dependent exploits go undetected. Intent and behavioral analysis of tooling is required to surface these gaps.
What is an agentic runtime sandbox and why does it matter?
An agentic runtime sandbox is an isolated execution environment that monitors and constrains what an agent can actually do during a run — not just what it was instructed to do. It captures the full Agentic Action Path: model decisions, tool calls, data access, and execution outcomes. Without a true sandbox, claims of 'runtime monitoring' are surface-level. A real sandbox enforces pre-execution controls at the point of action and provides the visibility needed to verify agent behavior against policy.
How can teams assess the trustworthiness of MCP servers and agent tools?
Teams need to assess and classify the tools, skills, and MCP servers their agents run — not just at onboarding, but continuously, to detect behavioral drift across versions. This requires intent and behavioral analysis rather than static code scanning. A trust registry approach — where MCP servers and tools are evaluated, scored, and monitored for changes in execution directives — gives teams a verifiable basis for deciding which components agents are permitted to invoke.
