AI agent governance: Safe deployment and operational controls

Run through this list. The one that matches is usually the actual blocker, not whatever the pilot team has been told:

• No agent inventory. Nobody can produce a complete list of the agents already running. This resolves in the pre-production readiness gate below.
• No agent identity. Agents run on a person's credentials or a shared service account, so actions can't be attributed to anything specific. This resolves in the section on governing what an agent can reach.
• Over-broad data access. The pilot got a wide grant to move fast, and nothing narrowed it before scale. This also resolves in the access section.
• No approval path. Nobody has decided which agent actions actually need a human, so security blocks all of them by default. This resolves in the section on where human approval belongs.
• No audit trail. Nothing can reconstruct what the agent did, so it can't pass review. This resolves in the audit logs and evidence section.

Here are the specific, testable conditions that must be true before an agent ships:

• The agent is registered and has a named owner
• It has its own identity, separate from any human's
• Its data access is scoped and inherited from source-system permissions
• Its available tools and actions are on an explicit allowlist
• A risk tier is assigned and the approval path is defined
• Logging is wired in before launch, not added after
• A rollback and incident response path exists

A named gate like this is more useful than a general list of principles, because a team can check each item off rather than debate whether they've "done governance" in the abstract.

This section covers agent identity and scoped access specifically, not a full data access controls guide. For deeper guidance on permission models and policy enforcement, see the Context Governance access controls content once published.

An access model worth building out as a table:

Agents should have their own identity rather than running through a human account or a shared service account, for a simple reason: if an action can't be attributed to a specific agent, it can't be audited, revoked or scoped independently of every other thing using that same credential.

Access should be scoped tightly before production and reviewed on a set cadence after launch, not granted broadly once and left alone. A pilot's wide initial grant is a normal, reasonable shortcut to get something working. It becomes a liability the moment it's still in place after the pilot proves out and the agent starts touching real production data.

A control-point table for the actions that matter most:

A few principles worth stating plainly:

• The allowlist principle. An agent gets an explicit list of permitted tools and actions. It never gets a default-allow grant that it's expected to self-limit within.
• Rate and scope limits matter early. An unbounded loop is a cost and blast-radius problem well before it's a quality problem. An agent that can call an expensive tool a thousand times in a runaway loop is dangerous regardless of whether any individual call was reasonable.
• Separate read from write explicitly. Read actions can often be scoped fairly broadly. Write actions, anything that changes state, should be narrow and individually justified.
• Some actions should never be automated, on any risk tier. Changing a permission and creating another agent both belong on this list. The blast radius of getting either wrong is large enough that a human should always be in that loop, regardless of how mature the guardrails around everything else are.

Guardrails in this sense aren't a separate concept from controls. They're the specific, enforced boundaries on what an agent is permitted to attempt, expressed concretely enough that a security reviewer can check them rather than take them on faith.

The weakest part of an organization's existing governance thinking is often tied to a process that lacks a human in the loop. Often, the human-in-the-loop step is used to describe data labeling and model training oversight, which is a related but different problem. The problem here is approving what an agent does in production, not in training.

A risk-tier model for approval:

A few operational details that determine whether this actually works:

• Name the role, not "a human." Approval needs a specific accountable role, not a vague reference to human oversight. And address directly what happens when the person who built the agent is also the one approving its actions: that's a conflict of interest, and the approval path should route around it for anything above the lowest risk tier.
• Set a timeout default, and state whether it fails open or fails closed. If an approver doesn't respond in time, does the action proceed anyway or does it get blocked? Recommend failing closed for anything above the lowest risk tier. A missed approval should not become a silent yes.

Designing approval that doesn't become rubber-stamping. Volume is the usual failure mode here: if a tier is sending an approver hundreds of requests a week, the tier is miscalibrated, not the approver. A useful test is simply what percentage of requests actually get rejected. Near-zero rejection is a strong signal that the approval gate is theater rather than a real control, and the fix is usually to reclassify the tier, not to lecture the approver about being more careful.

The audit link is important too: An approval is only a real control if the approval reference actually lands in the audit log. An approval that happens in a chat message or a verbal sign-off, disconnected from the record of the action it approved, isn't a control an auditor can verify later.

You can learn about AI model and agent monitoring and AI observability, where we cover drift detection, monitoring metrics and observability in depth. This guide covers monitoring specifically as an operational control within agent governance.

What's specific to agents, as distinct from general model monitoring:

• Action sequences, the order and combination of things an agent actually did
• Tool selection, which tools it chose to call and whether that matches expectations
• Scope creep, whether its actual usage is expanding beyond its registered purpose
• Denied actions, what it attempted that policy blocked
• Approvals, how often it's hitting each tier and how approvers are responding
• Cost, what it's actually consuming in compute or paid tool calls
• Unexpected behavior, anything that doesn't match its documented purpose

See our AI audit trails to get a detailed checklist for what to log. However, the one argument worth making explicitly here is that agent evidence should be captured at the control layer, not only inside the application running the agent.

An application-level log can be incomplete, inconsistent across different agents built by different teams or simply absent if the team building the agent didn't prioritize it. Evidence captured at the control layer exists regardless of how carefully any individual team built their agent.

A worked example of what a single logged agent action should contain:

• The request: What was asked, and by what triggering event
• Agent identity: Which specific agent handled it
• Permission check: What access was verified before proceeding
• Approval reference: Which tier applied and, if relevant, who approved it
• Action taken: Exactly what the agent did
• Evidence produced: The record tying all of the above together, timestamped and queryable later

For deeper checklists on what to log and how to prepare for an audit, see our blog on AI audit readiness.

Most existing content addresses registration and controls at the point of deployment. Far less of it addresses the phased path an organization actually needs to move from one working pilot to broad, governed production use.

A phased rollout sequence, each phase with a named exit criterion:

• Phase 1: One agent, fully governed. Pick the lowest-risk, genuinely useful agent available. Exit this phase when it has run for a defined period with complete logs and no out-of-scope actions.
• Phase 2: One use case class. Extend to other agents doing similar work under the same policy. Exit when the policy has needed no per-agent exceptions to hold.
• Phase 3: Write actions. Introduce agents that change state, not just read it. Exit when the approval tier for these actions is holding steady without a growing backlog.
• Phase 4: Broad rollout. Registration becomes a condition of deployment rather than a follow-up task teams remember to do eventually.

What to measure at each phase:

• Out-of-scope action count
• Approval rejection rate
• Time from request to approval
• Percentage of running agents that are actually registered

What to deliberately defer until the earlier phases hold: Multi-agent coordination, autonomous spend and anything customer-facing. None of these are wrong to eventually pursue. They're wrong to pursue before the foundational phases have actually proven stable.

The common, expensive mistake is running the pilot outside governance entirely, then trying to retrofit controls once it's already working and stakeholders are attached to shipping it fast. Retrofitting identity, access scoping and logging onto an agent that's already live costs meaningfully more than building it in from phase one, and it usually happens under more schedule pressure, not less.

Agent governance depends on a current registry, governed context and access, policy and approval workflows that actually connect to each other, and monitoring and audit evidence that get produced automatically. Most enterprises have some of these; very few have all of them tied together.

AI Command Center is Collibra's product built specifically for this. Every agent is registered as a governed asset. Access is scoped from source-system permissions rather than granted broadly and forgotten. Approval workflows tie to defined risk tiers with named accountable roles. Monitoring tracks agent-specific signals (action sequences, tool selection, scope creep, cost) continuously once agents are live. And audit evidence is captured at the control layer, so it exists regardless of how carefully any individual team built their agent.

Underneath AI Command Center, Collibra Platform provides the governed context foundation: definitions, lineage, ownership and policy attached to the data agents consume. That combination — governed context feeding runtime enforcement — is what turns agent governance from a security team's veto into an approval path pilot teams can actually pass.

See, trace and govern every AI agent in production with Collibra AI Command Center.

Learn more about Collibra Platform, the enterprise AI control plane.