Securing AI Agents for Real-World Action
As AI systems move from conversation into action, the security model around them has to change. Chatbots generate content. Agents exercise delegated authority — acting across files, tools, accounts, and workflows. This shifts the primary risk from incorrect output to incorrect action. At Anna, we believe useful agents must be built around this distinction. Capability alone is not enough. Trust comes from capability with boundaries. As a hybrid AI agent, Anna combines local execution with cloud intelligence. This architecture allows agents to work where users already work, while also making clear system boundaries essential. In this post, we outline the security principles behind Anna’s approach to building agents that can act safely in real-world environments.
Summary
As AI agents become more capable, their security model must account for more than model behavior. Agents need safeguards around the authority they use, the data they process, and the actions they are allowed to take.
For Anna, this means treating these three elements as core infrastructure boundaries: credentials should be mediated rather than exposed directly to apps; context should move through the system in scoped and minimized ways; and sensitive constraints should be enforced by the system, not left only to prompt instructions.
These measures can begin as product design principles, but over time we believe they will become part of the broader trust standard for AI agents. Trustworthy agents will not be the agents with the broadest access. They will be the agents whose power is constrained by design.
From content safety to action safety
Most AI safety discussions focus on model outputs: whether a system says something harmful, misleading, or incorrect. Those remain important questions. But agents introduce a different layer of concern: action safety. An agent may summarize a document, but it may also move files, send emails, or operate across multiple tools. Each action carries authority, and each raises boundary questions: what the agent can access, what it can use, what data can leave the system, and which actions require approval. We think these questions should be treated as infrastructure concerns, not interface details. Anna’s security model is organized around three boundaries: authority, data, and execution.
Authority boundaries: credential isolation
For agents to be useful, they need to operate through the tools and accounts users already rely on. This makes credential handling one of the central security questions for agent systems.
The unsafe approach is to hand credentials directly to application code. Anna is designed around a different model: runtime credential mediation. Third-party apps should not directly receive or store user credentials. Instead, they request controlled tool execution, and credentials are injected only at the moment of execution inside the trusted tool layer. This allows an app to perform an authorized operation without being able to extract the credential itself. In practice, this means the agent can use authority without owning authority.
As agent ecosystems become more open, this distinction becomes increasingly important. AI-native apps will often be powerful because they compose multiple tools and workflows. The more powerful the app, the more important it becomes to minimize credential exposure.
Our goal is to make credentials operationally useful while keeping them structurally constrained.
Data boundaries: controlled context flow
Because Anna uses a hybrid architecture, context flow becomes a security question as well as a performance question.
In the basic case, Anna prepares context locally — selecting, trimming, summarizing, and structuring task-relevant information — then sends a scoped version to a cloud model for inference. The harder question is what happens when context is too large for a single prompt. Sending entire files, local scans, or tool outputs to the model is inefficient and often unnecessary. For these cases, Anna separates stored context from model context. It stores controlled data fragments and sends the model only a brief, an index, or the relevant subset needed for the task.
The security requirement is explicit layering: distinguish local context, model context, and stored fragments; reduce unnecessary data movement; preserve user and tenant isolation; and scope retrieved fragments to the workflow that created them.
Execution boundaries: system-level enforcement
Execution boundaries control what the agent is allowed to do.
It is important to note that prompts are not security boundaries. Prompts can guide behavior, reduce errors, and express policy in a form the model can reason about. But a prompt is not an operating-system permission, a runtime guard, or a code-enforced access rule. A secure execution boundary requires both model-level instruction and system-level enforcement. The principle is simple: the model can propose; the system must enforce.
Anna’s Sandbox Mode is built around this principle. When Sandbox Mode is enabled, Anna can be limited to a user-specified folder or workspace. If the agent attempts to access files outside the permitted path, the operation is blocked.
For users, this means they can define a working area, grant bounded access, and trust that the agent cannot casually wander across the rest of the machine.
Toward an agent security model
Traditional software security often assumes that applications are tools operated by humans. AI agents change that assumption. They are active systems that interpret intent, select tools, transform context, and take steps on the user’s behalf.
That shift requires a security model that treats agency itself as the object of control. For Anna, this means separating what an agent can reason about from what it can access, what an app can request from what credentials it can see, and what a model is instructed to do from what the system is allowed to permit.
This work is early. As agents become more capable, these boundaries will need to become more precise, more visible to users, and more deeply enforced across the system.
Conclusion
It can be tempting to measure agents by how much they can access and how much they can do. But for agents to become trusted participants in real work, power alone is not enough.
The future of AI agents will not be determined only by model capability. It will also be determined by whether users can safely delegate real work.
AI agents will become a new layer of computing. They will help users work across tools, files, accounts, and devices. For that to be viable, they must be built not just to act, but to act within well-defined boundaries.
Trustworthy agents will not be the agents with the broadest access. They will be the agents whose power is constrained by design.