AI agent governance is the set of controls that decide what AI agents may do, enforce those decisions while the agents run, and prove afterwards what they did. In practice it covers seven things: an inventory with real identities, least privilege, runtime enforcement on every action, human oversight for risky steps, audit evidence, a kill switch and budgets. Agentic AI governance differs from general AI governance because agents act, not just answer.
What does AI agent governance mean?
Traditional AI governance asks whether a model is fit for purpose: is it accurate, fair, documented and approved? Those questions still matter. But an agent also calls tools, changes records, sends messages and spends money on someone's behalf. Governance for agents adds a second question: is this particular action, by this agent, for this person, allowed right now? That question has to be answered thousands of times a day, by software, before each action happens.
Why do AI agents need their own governance?
Three properties make agents harder to govern than models or ordinary applications:
- They are steered by text. An agent decides what to do from its prompt and from everything it reads. A web page or an email can change its plan. The OWASP Top 10 for Agentic Applications lists agent goal hijack as its first risk.
- They act with delegated authority. An agent often uses a person's or a team's credentials, so a mistake has that person's reach.
- Their code is not reviewed. Agents run tool sequences and sometimes code that nobody wrote in advance.
The safe assumption is that the model will sometimes be wrong or manipulated. Governance has to hold even then, which means it must sit outside the model.
What controls does AI agent governance need?
- Inventory and identity. Know every agent, what it runs, which tools and models it can reach, and give each run its own identity that names the person it acts for. Shared service accounts make audit impossible.
- Least privilege. Each agent gets only the tools it needs, with short-lived credentials. Anything not explicitly allowed is denied.
- Runtime enforcement. A control point checks each action against policy before it runs, including the arguments. Checks inside the agent's own process can be skipped by a manipulated agent.
- Human oversight. Risky actions wait for a person, with separation of duties. See human-in-the-loop approval for AI agents.
- Audit evidence. A record of every decision, including denials, that you can show has not been edited. See tamper-evident audit trails.
- A kill switch. A way to stop one agent, one user's agents or everything, within seconds. See the AI agent kill switch, explained.
- Budgets. Hard limits on spend per session, day or month, so a loop cannot run up an unbounded bill.
AI agent governance checklist
Use this table to assess any setup, whether you build it yourself or buy it.
| Control | Question to ask | Good answer | In MAQPNA |
|---|---|---|---|
| Inventory | Can you list every agent, its tools, models and policies? | Generated from the running system, not a spreadsheet | AI-BOM export (CycloneDX or SPDX) |
| Identity | Does each call say which agent, which session and which person? | A short-lived, signed identity per run | Session token with the person in act, 15 minutes by default |
| Least privilege | What happens when no rule matches? | Deny | Default deny; scopes per tool server |
| Runtime enforcement | Can the agent skip the check? | No: enforced outside its process, on arguments too | Gateway policy per tool and argument; sandbox whose only exit is the gateway |
| Human oversight | Can risky calls wait for two people who are not the requester? | Yes, bound to the exact call | Approvals with four-eyes, no self-approval, user confirmation (CIBA) |
| Audit evidence | Can you prove the log was not edited? | Hash chain with signed checkpoints, verifiable offline | Audit ledger; signed evidence packs in paid editions |
| Kill switch | How long does it take to stop one agent everywhere? | Seconds, enforced at the control point | Revocations at the gateway |
| Budgets | What stops a runaway loop's spend? | Limits checked before the next call | Budgets per agent, namespace, user or tenant |
What does Singapore's IMDA agentic AI framework add?
Singapore's Infocomm Media Development Authority (IMDA) published a Model AI Governance Framework for Agentic AI in January 2026, one of the first government frameworks written specifically for agents, and updated it in May 2026 with more on human approvals and access controls (Baker McKenzie summary). It is voluntary guidance, not law. Its themes map closely onto the controls above: bound what an agent may do before it runs, keep people meaningfully accountable for its actions, apply technical controls such as least-privilege access and human checkpoints for significant actions, and keep the records that let an organisation show what happened. If you follow it, the runtime controls are where its expectations become checkable.
How do AI governance platforms relate to runtime enforcement?
"AI governance platform" usually means software for the paper side: a registry of models and use cases, risk assessments, approval workflows, documentation and mapping to frameworks such as the NIST AI Risk Management Framework or ISO/IEC 42001. That work matters, and regulators expect it.
Runtime enforcement is the other half. It sits in the path of each action and applies the rules the governance process agreed. A registry can record that "the support agent may issue refunds up to 100 euros"; only something in the request path can refuse a refund of 5,000. The two meet in the evidence: the runtime's audit trail shows that the documented policy was actually enforced.
| AI governance platform | Runtime enforcement | |
|---|---|---|
| Works on | Models, use cases, risks, documents | Individual tool calls, model calls and connections |
| When | Before deployment and at review points | Every time an agent acts |
| Output | Assessments, approvals, reports | Allow, deny or hold decisions, and an audit trail |
| Fails if missing | No accountability or documentation | Rules exist but nothing applies them |
Where does MAQPNA fit?
MAQPNA is the runtime enforcement half. It is a self-hosted runtime that runs each agent session in its own sandbox and puts every tool call, model call and egress connection through one gateway. The gateway checks identity, policy, DLP and budgets, holds calls for approval when a rule says so, and writes a tamper-evident ledger. It hosts agents built with any framework, and it does not replace a governance platform or model evaluation. The overview in the docs explains the parts.
For regulation, MAQPNA's ledger and approvals support evidence for record-keeping and human-oversight duties; they do not make you compliant on their own. See the EU AI Act and AI agents and the OWASP agentic mapping. For a concrete failure case, read how to stop an AI agent deleting a production database. For governance where nothing may leave your network, see sovereign, air-gapped agent governance.