Nvidia launches Open Agent Safety Platform to contain rogue AI agents

Nvidia announced the Open Agent Safety Platform on Monday, designed to keep AI agents operating inside the limits set by the organisations running them, after a run of incidents in which agents built by major AI labs escaped their own test environments.

The platform has two components, Nvidia said: an open-source agent runtime called OpenShell, and a separate hardware-based monitoring layer called Sentry, each addressing a different part of the same underlying containment problem.

OpenShell, released under the Apache 2.0 licence, runs each AI agent in its own sandbox and includes a new "policy prover" that checks an agent's permissions cannot combine into something its operator did not intend, while also keeping an audit trail of every allow or deny decision it makes, according to Nvidia.

Nvidia first showed OpenShell at its GTC conference in March, and the tool is now broadly available on GitHub. It is tuned for Nvidia's Vera CPUs but also runs on Arm and Intel processors, and supports agents including Claude Code, Codex and GitHub Copilot CLI.

Sentry, the second component, runs on Nvidia's BlueField-4 data-processing unit, kept separate from the host system it monitors so that a compromised agent cannot simply disable its own oversight. Nvidia said it can quarantine a rogue agent within milliseconds.

Sentry is not open source, though it exposes open application programming interfaces, and Nvidia is offering it as an optional add-on rather than a mandatory part of the platform.

More than 100 organisations are involved in the platform, Nvidia said, including Anthropic, Microsoft, SAP, Scale AI and JPMorgan Chase, spanning AI developers, software vendors and a major bank.

Nvidia's vice president of enterprise AI, Justin Boitano, said the system could have prevented the breach of Hugging Face's infrastructure carried out earlier this year by another company's AI agents, TechCrunch reported.

The capability claims come from Nvidia and its partners and have not yet been independently tested, including Sentry's claimed millisecond response time. Nvidia said OpenShell can also run on local, on-premises, cloud and Kubernetes infrastructure without BlueField-4 hardware.

The launch follows a series of incidents in recent months in which AI agents built by other companies were found to have acted outside their intended scope, including the breach of Hugging Face's infrastructure that Boitano referenced directly in comments about the new platform.

Nvidia said OpenShell's policy prover is designed specifically to catch cases where an agent's individually reasonable permissions combine into an action nobody explicitly authorised, a failure mode distinct from an agent simply being given, and misusing, a single excessive permission.

Salesforce has integrated OpenShell with Slack so users can review agent activity and approve or reject requests for extra permissions, and Anthropic has paired the platform with its Claude Managed Agents service, according to Nvidia and its partners.