Unit Tools

AI safety claims face executive scrutiny

 ·  By Thalia Whitmore
AI safety claims face executive scrutiny - ai safety
AI safety claims face executive scrutiny

Enterprises have spent decades refining how they audit people and software. Agentic AI introduces a third category—systems that interpret instructions, call tools, and act across workflows without a mature assurance model.

Autonomy outpaces accountability

Agentic AI complicates oversight because a single outcome may pass through multiple systems. An agent can collect data, select a tool, generate code, route a request, and hand off work to another agent before anyone approves the final result. No single manager may see the full path.

Executives remain accountable even as operating activity grows more autonomous. The CIO must explain who authorized an action, whether the agent stayed within its approved purpose, and what evidence supports that claim.

In February 2026, NIST launched an initiative focused on secure operation and interoperability for agents capable of autonomous action. The move reflects concern that adoption is accelerating faster than governance can keep up.

One study predicted that 40% of enterprise applications would include task-specific agents by the end of 2026, up from less than 5% in 2025. However, over 40% of these projects could be canceled by the end of 2027 due to rising costs, unclear business value, or weak risk controls.

A 2025 report found that 13% of surveyed organizations experienced breaches involving AI models or applications. Among those, nearly all cited inadequate access controls. Another 63% lacked policies for managing AI or preventing unauthorized use.

Related: EU demands redesign of addictive Instagram and Facebook

Two-thirds of CIOs and CTOs said they were held responsible for AI systems they did not fully control. Most noted that technology was spreading across the business faster than IT could track it, while 77% said adoption was outpacing governance capabilities.

Management carries accountability while control remains distributed across teams, systems, and workflows. An assurance model must prove where boundaries held rather than just state intent.

Assurance requires more than policies and logs

Before deployment, management should document an agent’s business purpose, its owner, the systems it touches, allowed actions, and stop conditions. Testing should determine whether the agent can access information outside its scope, call an unapproved tool, continue after a stop condition, or carry incorrect assumptions into another system.

These systems operate in environments that change over time. A control approved during deployment can weaken months later without an obvious change to the application. NIST’s work on monitoring deployed AI highlights drift, fragmented logging, and immature standards as barriers to oversight. The World Economic Forum recommends scaling safeguards alongside an agent’s autonomy, authority, and complexity.

For every agent with meaningful operating authority, four records should exist: the approved baseline, boundary-test results, a history of behavioral drift, and an account of exceptions and interventions. Together, they provide a defensible record for board discussions, audits, or regulatory responses without relying on technical teams.

Consider a coding agent that starts by drafting test cases, then gains access to repositories, tickets, CI/CD tools, and production documentation. A single production change could involve an instruction, code, a tool call, an automated test, a ticket update, and human approval. Assurance must show how the result was produced, which systems participated, whether the agent crossed a boundary, and how exceptions were handled.

This method resembles documentation requirements for critical software. The difference lies in layers of interpretation and decision-making that traditional systems lack. A static policy won’t work when an agent’s behavior can shift based on model updates, prompt changes, or new data connections.

Related: Snowflake Introduces AI Cost Controls

Autonomy should scale only as fast as verification

Internal teams design controls and operate the environment. At the board level, management also needs review independent from those who built and run the agent.

Assurance must be part of the workflow. The operating record should capture approved purpose, tested boundaries, material changes, exceptions, human interventions, and unresolved findings. Independent review can then examine whether controls operated as intended.

Has the agent’s behavior remained within tested boundaries?

Exceptions must be resolved, or they may indicate a systemic issue. The operating record should provide a clear, auditable trail of decisions and interventions.

Those standards help decide whether an agent is ready to move from a limited workflow into broader operations. Drift or repeated interventions should pause expansion until the cause is understood.

Agentic AI needs its own assurance model—one that accounts for approved behavior, tested boundaries, monitored change, and documented intervention. Without it, enterprises deploy systems they cannot fully explain, control, or trust. Businesses integrating enterprise AI strategies must address these gaps to avoid unintended consequences.

Leave a Comment

Your email address will not be published.