Skip to content

Blog · AI Implementation

AI Agent Governance for a Small Team: Six Decisions to Write Down

How a small team can govern AI agents with six written decisions, where an AI usage policy stops, and when frameworks such as NIST AI RMF and ISO/IEC 42001 are worth the time.

  • The Foregrounds
  • 16 min read
A bone-white chalk drawing of a six-port desktop connection hub with a single blue disconnect rocker on a black background.

On May 20, 2026, Singapore's Infocomm Media Development Authority (IMDA) published version 1.5 of its Model AI Governance Framework for Agentic AI, with feedback from more than 60 companies. One line bears directly on a team whose first agent now reads client mail and drafts messages or files documents on its own: "Enforce human approval through system-level controls where possible, vs prompt-layer guardrails, which may be bypassed or 'forgotten'." A prompt-layer guardrail is a rule in the agent's instructions, and in a small team that sentence may be the only limit the agent has.

The goal of AI agent governance is to be able to say, for each agent, what it may do and who answers for it. Teams can miss it in two opposite ways, by treating the usage policy and the agent's instructions as enough, or by copying a large company's committee structure and never finishing it. In Deloitte's April 2026 account of its survey of 3,235 information technology and business leaders from 24 countries, only 21% said their organizations have a mature governance model for agentic AI. This article sets out six decisions that fit on one page per agent, applies them to a public case, and says where frameworks and certificates come in.

AI agent governance covers what software may do, and an AI usage policy covers what people may do

A usage policy, perhaps adapted from an AI policy template, tells staff which tools they may use and what they may put into them. Agent governance sets limits for software that acts for you. Microsoft's guidance draws a similar line, saying that responsible AI policies "define acceptable use of AI across the organization" and that agents "operate with delegated authority and can affect multiple systems at once." A policy addressed to people therefore cannot, on its own, restrict what a mailbox connection allows.

Agents can also arrive unplanned. A Microsoft security post says 29% of employees have turned to unsanctioned AI agents, citing a July 2025 survey of more than 1,700 data security professionals that Microsoft commissioned from Hypothesis Group. Those respondents were security professionals, not the employees whose use the figure describes, and Microsoft sells agent governance products, so treat it as a rough indication rather than an estimate for small teams. Our recommendation is one rule in your policy: no agent connects to client systems until its limits are written down and enforced.

An agent register is a one-page record for each agent that says what it may do alone, what stops for a person, what it may never touch, who owns it, when it gets reviewed and how to stop it. Microsoft's guidance asks for something similar, an agent registry that tracks ownership, purpose, platform and access scope. The page described here adds the limits and the stop, and a small team can keep it by hand, since the same guidance says "If the environment is small, manual tracking might be sufficient for early adoption." A hypothetical agent that handles a new client's onboarding pack at an advisory practice shows what each line might say.

Six tiles name the lines of an agent register: may do alone, stops for a person, may never touch, owner, review and stop.
The six lines of an agent register. The never-touch line is the one a written page cannot enforce by itself. View full-size image

1. Decide what the agent may finish on its own

Start with low-risk actions whose mistakes you could see and put right the same day, because reversibility matters alongside the harm a mistake could cause. IMDA compares an agent that schedules meetings, which "can easily reschedule them if an error is made", with one that sends email to external parties. The onboarding agent fills in the checklist and creates internal tasks on its own. If you have worked out which steps need a rule, a model or a person, this line records the result for the whole agent.

2. Decide which actions stop for a named person

Some actions cannot be taken back, and IMDA lists them among its approval checkpoints: "permanently deleting data, sending communications, making payments." Put a role and a person beside each action on your list, because a request sent to a whole team can sit unanswered. IMDA advises "Denying action by default" when approvals fail, including when human supervisors are unreachable, so if the approver is away the agent waits and a named backup picks it up. The welcome email, for instance, waits for the account lead. An inbox agent needs more detail, so write the action-by-action permission list for an inbox agent and copy the result here.

3. Decide what the agent may never touch, and block it in the system

A limit on this line counts only when the connection enforces it, so the work here is removing access. IMDA's advice is direct: "rather than relying on prompts to instruct the agent against accessing certain tools, impose access controls that prevent the tool from being called by the agent at all." It also suggests that you "attempt disallowed actions to ensure restrictions work", so try each one with test data, treating a single passed check as a starting point. For example, the onboarding agent has no access to other clients' folders or to billing, so a wrong instruction has nothing there to reach.

4. Decide who owns the agent, by name

One named person answers for the agent's limits, reads its exceptions, meaning the cases it could not handle within those limits, and can stop it. When that duty belongs to several people, each of them can assume that someone else is reading the exceptions. Choose an owner who has time to read those cases and act on them, such as the operations lead for the onboarding agent. Agree a backup and arrange a handover if the owner leaves, because Microsoft's admin documentation notes, for agents created with its Agent Builder, that "Agents become ownerless when their original creator leaves the organization."

5. Decide when the agent gets reviewed, and what triggers an early review

People stop looking closely at software that has behaved well, which IMDA describes as "the tendency to over-trust an automated system, especially when it has performed reliably in the past." It suggests tracking how often people reject or change the agent's actions, and treats a low rate as a possible sign that approvers have stopped checking.

So we suggest a monthly review at first, in which the owner reads the exceptions and what people changed. Consider quarterly reviews only after several have confirmed that people are still checking and that the limits remain appropriate. Any new connection, new model or larger scope should trigger an early one, as it would if someone proposed connecting the onboarding agent to another system. This review checks the limits, and whether the agent paid off is a separate question.

6. Decide how you will stop it, and try it once

A stop is a set of steps a person follows to cut the agent's access, and none of them should depend on the agent reading a message. Microsoft's guidance says to "Decide in advance how to quickly disable an agent if it malfunctions or causes harm", and it suggests drills where an agent supports critical operations. We recommend trying the stop once for any agent that acts for clients, while nothing is at stake, and timing it. For the onboarding agent, the operations lead removes its mailbox and document connections. Stopping an agent does not undo what it already did, so note where its log lives.

A chalk drawing of one page for the onboarding agent with all six lines filled in and a callout marking the never-touch line as enforced in the connection.
The onboarding agent's page with all six lines filled in. Check the never-touch line against what the connection allows. View full-size image

The register applied to a public case: the OpenClaw inbox incident

One public account lets us apply these six lines to a reported incident. On February 23, 2026, TechCrunch reported on an account by Summer Yue, whom it describes as a Meta AI security researcher. She had asked an OpenClaw agent, an open-source personal assistant, to check her overstuffed inbox and suggest what to delete or archive. By her account, it began deleting email and ignored stop commands from her phone, and she wrote, "I had to RUN to my Mac mini like I was defusing a bomb."

Yue believes the large inbox "triggered compaction", which is when an agent summarizes a conversation that has grown too long, and that it may have skipped her prompt telling it not to act. TechCrunch "could not independently verify what happened to Yue's inbox", and the AI Incident Database, which logs this as Incident 1542, just says the agent "attempted to delete" emails. Asked whether she had been testing guardrails on purpose, she replied, "Rookie mistake tbh." She reported her own mistake in public, which is why others can learn from it. Here are the six lines as they read against what those two sources report:

  • May do alone: reading and suggesting, which is what she asked for.
  • Stops for a person: deleting and archiving. By her account, the agent ignored an approval instruction in the conversation, illustrating the risk described in IMDA's advice on OpenClaw deployments.
  • May never touch: the reports do not describe any separate restrictions on data or systems. The agent could evidently delete, and we infer that a read-only connection would have prevented it, though the reports do not describe its permissions.
  • Owner: Yue herself, who was present and works in the field. The deleting still continued until she reached the machine, so a watchful owner does not replace a limit in the system.
  • Review: she had tested on a smaller "toy" inbox, where it did well. Moving to her full inbox changed the scope, which should prompt a fresh review.
  • Stop: messages to the agent did not stop it. The Incident Database says she reportedly terminated it from the Mac mini hosting it.

It’s worth calling out some caveats. This is one person's account that TechCrunch could not independently verify, and the cause is her belief, not an established finding. It involved a personal inbox and an open-source assistant, with no client involved. The sources do not say whether messages were recovered, how often this happens, or whether a commercial product would behave the same way. So the case illustrates what each line asks you to check, and it says nothing about how likely this is with your own agent.

What AI governance frameworks such as NIST AI RMF and ISO/IEC 42001 are for, and when a small team needs them

Both are useful, depending on the scenario. The NIST AI Risk Management Framework, from the US National Institute of Standards and Technology, is "intended to be voluntary", so you can treat it as a reference for checking what your register leaves out, knowing that NIST says it "is being revised." NIST's agent initiative page, updated August 14, 2026 and checked on November 1, lists a request for information, a draft concept paper and listening sessions, but does not list finished guidance for deployers. That may change, so check the page again when you review your register.

ISO/IEC 42001 is "applicable to any organization, regardless of size, type and nature", but it sets requirements for an AI management system: the policies, responsibilities and processes your organization uses to manage AI. A certificate means that system has been independently assessed. As we read it, you need the certificate when a client, a contract or a bid requires one. Selling AI systems is a reason to consider the standard, but it does not by itself create a need for a certificate. A vendor's certificate does not certify your own organization, and Microsoft says of its certificate that you remain responsible for "engaging an assessor" for your own organization.

The Foregrounds holds no compliance certifications, as our FAQ says. What we write instead is a safety contract that covers the same ground as lines 1 to 3: what the agent may complete alone, what must stop for a named approver and what it may never touch. Our job is to know what current tools can and cannot do, and to adapt the limits as those tools change.

Questions to ask about your own agents

These questions show where your own pages have gaps:

  • Can you name every agent that acts in client mail or a client system, including features switched on inside tools you already pay for?
  • Which limits exist only in the agent's instructions?
  • When did someone last read what the agent declined or got wrong?
  • Who could cut its access within ten minutes, and have they tried it?

The first step to take this week

Pick the agent closest to your clients, book thirty minutes with whoever set it up, and fill in the six lines together. Then open the connection's permissions and mark every line where the page and the settings disagree. That list tells you what to fix first, starting with anything the connection allows that line 3 forbids.

Questions buyers ask

Do we need an AI committee before our first agent?

No, not for one or two agents, because a committee helps when many teams need a common answer. Microsoft's guidance says "If agent usage is limited, existing governance forums might be sufficient", so add a standing item to your leadership meeting. Look again when agents run "across multiple business units", which is where Microsoft says to formalize accountability.

Do our client contracts need to change when an agent handles client work?

That depends on what they already say, so take the question to your counsel, because this article is not legal advice. Requirements differ by profession and client, and a contract may already limit subcontractors or the tools that handle client data. To prepare, mark the clauses on confidentiality, subcontractors and AI in your largest client contracts, and bring counsel that list with the agent's page.

Who is liable if an agent makes a mistake?

That depends on your contracts, your insurance and where you operate, so take it to your counsel and your insurer, because we cannot give legal advice. IMDA says "organisations and humans remain accountable", which is Singapore's guidance and not a legal ruling. To prepare, gather the vendor contract and the agent's page, and ask your insurer in writing whether your policy covers errors made by an agent.

How do we govern agents inside vendors' products we did not build?

Give each agent in a vendor's product a page too, and turn any gap on it into a question for the vendor. IMDA warns that agents from outside providers "can be difficult to observe and control", and it advises checking whether the product records the tools an agent uses and the data it accesses. List the agent features switched on in your tools, ask each vendor who can act, what is logged and how to turn it off, and switch off anything nobody will own.

Each agent needs one page of written limits, one named owner and connections that enforce those limits, and those give your team a practical place to start. To plan and build an agent that way with your team, see our work on agentic systems.

Start withthe businessproblem.

Thirty minutes. Tell us what's slow or manual today, and we'll tell you honestly whether we can fix it.

30 minutes