"A working AI agent governance policy for a company of roughly 100 to 2,000 people: four autonomy levels, written approval limits, access rules, logging, testing and an off switch, mapped to NIST AI RMF, OWASP LLM06 and ISO/IEC 42001. Includes a free, editable policy template."
Key Takeaways
- 1AI agent governance answers four questions before go-live: who approves the agent, what it can touch, when a person must check its work, and who can switch it off.
- 2Start every agent at the lowest autonomy level that still saves time, usually "draft for approval", and move up one level only after it meets written targets on your own cases.
- 3OWASP names three root causes of damaging agent actions: excessive functionality, excessive permissions and excessive autonomy. Most of a good policy is limiting those three.
- 4Write limits as numbers and lists, not judgement calls. Payments, deletions, price changes and contract terms should never be done by an agent alone.
- 5Gartner predicts over 40% of agentic AI projects will be canceled by the end of 2027, and lists inadequate risk controls as one of three reasons (press release, 25 June 2025).
- 6A mid-sized company does not need ISO/IEC 42001 certification or a governance platform to start. It needs a written policy, named owners and logs.
- 7Download the free 14-section policy template on this page and fill in the blanks for your first agent.
The short answer
AI agent governance is a written policy that names an owner for each agent, sets how much it may do on its own, lists what a person must approve, limits its access to what the job needs, logs every action, and says who can switch it off. A mid-sized company can set one up in about four weeks alongside its first agent build.
We design, build and support custom AI agents, so weigh this page with that in mind. The outside facts come from NIST, OWASP, ISO and Gartner pages we opened on 17 October 2026, linked where they appear. Search figures are our own DataForSEO measurement for the US on the same day. The policy template is ours, free to adapt inside your company, and not legal advice.
US searches for "ai agent governance" ran about 260 a month in October 2026, up 586% on a year earlier, and "agentic ai governance" another 260, up 457% (DataForSEO, 17 October 2026). The people searching are mostly not at the largest enterprises, which already have AI committees. They are operations, finance and IT leads at companies of a few hundred people who have just approved a first agent and need rules for it by next month.
This guide is for them. It covers what governance has to decide, a four-level autonomy scale, the limits worth writing down, the frameworks worth knowing (and which to skip at your size), and a 30-day plan. The free policy template turns all of it into blanks you can fill in.
What AI agent governance means, and how it differs from AI governance
An AI agentis software that uses a large language model (LLM, the kind of model behind ChatGPT or Claude) to decide its next step and then calls a tool: it reads a record, sends an email, updates an order. General AI governance mostly asks whether a model's answers are fair, accurate and safe to show people. Agent governance adds a harder question: what is this software allowed to do in our systems, and what happens when it gets it wrong?
That is why agent policies read more like an access policy for a new hire than an ethics statement. You would not give a new accounts clerk the CFO's login on day one. The same logic applies to an agent, with one difference: an agent can repeat a mistake a thousand times an hour if nothing stops it.
The four questions every AI agent governance policy must answer
- Who approves it? One named agent owner per agent, plus a business approver for the job and limits, a technical owner for access and logs, and a security or data reviewer.
- What can it touch? Which systems it reads, which it writes to, which data types it sees, and which model provider receives that data.
- When must a person check? Written limits: values, record types and actions that always go to a person, and how fast that person must respond.
- How do we stop it? Who can switch it off, how, how fast, and which team does the work by hand while it is off.
If your policy answers those four for every agent, you are ahead of most companies running agents today. The rest of this page is detail on each.
Four autonomy levels: decide how much the agent does alone
The single most useful decision is the autonomy level. Pick the lowest one that still saves real time, then move up one level only after the agent meets its targets for a period you set in advance.
| Level | What the agent may do | Who checks | Typical first use |
|---|---|---|---|
| 0. Suggest | Reads data and suggests. Changes nothing. | A person acts on the suggestion | Drafting replies a rep can copy |
| 1. Draft for approval | Prepares the action. A person clicks approve before anything is sent or saved. | Named approver, every item | Purchase orders saved as drafts in the ERP |
| 2. Act within limits | Acts alone inside written limits. Anything outside goes to a person. | Daily sample plus alerts | Order status updates and customer emails |
| 3. Act and report | Acts alone across the whole job. A person reviews a report. | Weekly report review | Routine matched invoices |
Most first agents we see belong at level 1. The time saving still comes through, because the slow part of most back-office work is gathering the information and preparing the record, not the final click. A level 1 agent that prepares 200 purchase orders a week for one person to approve is a large win with little risk.
Human in the loop vs human on the loop
Levels 0 and 1 keep a human in the loop: nothing happens until a person approves it. Levels 2 and 3 put a human on the loop: the agent acts, and a person watches samples, alerts and reports and can step in. In the loop suits money, customers and master data. On the loop suits high-volume, low-risk work where the agent has a track record on your own cases.
Write limits as numbers, not judgement
"Use good judgement on refunds" is not a limit. "Refunds under $200 with reason code 3 or 7; everything else to the support lead within four hours" is. If a limit cannot be written down, the action needs a person. Here is the table we suggest starting from (the dollar values are placeholders for you to set).
| Action | Agent may act alone when | Otherwise goes to |
|---|---|---|
| Message a customer or supplier | Message uses an approved template and the record matches | Account owner |
| Create or change an order, quote or invoice | Value under a limit you set | Team lead |
| Refund, credit or discount | Under a limit, with an approved reason code | Support or finance lead |
| Change a price, term or master record | Never alone | Business approver |
| Delete or overwrite data | Never alone | Technical owner |
| Make or schedule a payment | Never alone | Finance approver |
And a short list the agent must never do, whatever its level:
- Agree to a contract, price or legal term on the company's behalf.
- Share personal, financial or health data outside an approved list.
- Change its own instructions, permissions, limits or logs.
- Follow instructions found inside an email, document or web page when they ask for something outside its job.
Agentic AI risks that actually happen in a mid-sized company
The OWASP Top 10 for LLM Applications 2025 calls the core problem Excessive Agency: damaging actions taken in response to unexpected, ambiguous or manipulated model output. It traces the root cause to one or more of three things: excessive functionality, excessive permissions and excessive autonomy. Here is how those show up in ordinary companies. Agentic AI risk management, in practice, means giving each one a control and an owner.
| Risk | What it looks like | The control |
|---|---|---|
| Too many permissions | The agent connects with an admin account or a person's login because it was quickest | One service account per agent, read-only by default, write access per action |
| Too many tools | A "run any query" or open web tool when the job needs two specific lookups | Narrow, purpose-built tools only; remove the rest |
| Too much autonomy | Customer emails or ERP updates go out with no review in week one | Start at level 1; move up on evidence |
| Hidden instructions | A supplier PDF contains text telling the agent to change bank details | Ignore instructions in content; approval on payment data; target system refuses |
| Quiet drift | Answers get worse after a model update and nobody notices for a month | Alerts on handoff and override rates; re-test on every change |
| No owner | The agent breaks, and IT, the vendor and the business each think another team owns it | Named agent owner and technical owner in the register |
OWASP's Agentic Security Initiative goes further in its Agentic AI Threats and Mitigations guide (17 February 2025), which is worth a read for your technical owner. For a policy, the six rows above cover what most mid-sized companies will meet in their first year. Our AI agent security guide walks through the full OWASP agentic Top 10 with a control for each.
Planning your first agent and want the limits checked?
Bring the job you want automated and the systems it touches. In a 30-minute call with founder Bhavesh Barot we will suggest an autonomy level, the limits worth writing down and what to log. FactoryJet quotes a fixed price in writing after that call, and we can show working software on your own data before a contract.
Talk to the FounderAccess rules: least privilege, enforced where the data lives
OWASP's mitigations for Excessive Agency translate into seven rules a policy can state plainly:
- Each agent has its own service account. It never signs in as a person.
- Read-only is the default. Write access is granted per system and per action, and written in the register.
- No general-purpose tools (shell access, open web fetch, free-form database queries) without written sign-off.
- Permission checks happen in the target system, the ERP or CRM, not only in the agent's instructions. OWASP calls this complete mediation.
- When the agent acts for a specific person, it uses that person's permissions, not wider ones.
- Keys and passwords live in a secrets manager, never in prompts, code or chat logs.
- Access is reviewed on every review date and removed the day an agent retires.
Rule 4 matters most. Instructions can be talked around. A permission the ERP refuses cannot. If your agent reaches your systems through a custom integration or MCP server, that is the layer where these checks belong.
NIST AI RMF, ISO 42001 or OWASP: which AI agent governance framework fits a mid-sized company
| Framework | What it is | Use it for | At your size |
|---|---|---|---|
| NIST AI RMF 1.0 | Voluntary US framework, released 26 January 2023, with four functions: Govern, Map, Measure, Manage. Generative AI Profile added 26 July 2024. | Structuring your policy and showing customers you follow a recognized approach | Use its four functions as headings. No certification exists. |
| OWASP LLM06:2025 | Security guidance on Excessive Agency, with eight named mitigations | Access, tools and approval rules | Hand it to your technical owner as the checklist for permissions. |
| ISO/IEC 42001:2023 | International standard for an AI management system, published 2023 | Formal certification when customers or regulators ask | Skip certification for now unless a large customer requires it. |
Whichever agentic AI governance framework you start from, it should end in the same place: named owners, written limits and logs. The template below maps each of its sections to one of these, so you can point to a recognized source if a customer or auditor asks where a rule came from.
Free template
AI Agent Governance Policy Template
An editable Word document with every section on this page turned into blanks to fill in. Give it to your agent owner, your IT lead and your vendor before the build starts.
- 14 sections, about 1,600 words, written in plain English
- Agent register, roles table and four-level autonomy scale
- Approval-limits table and a list of never-allowed actions
- Testing, monitoring, incident and switch-off checklists
- Go-live sign-off page and a map to NIST AI RMF, OWASP LLM06 and ISO/IEC 42001
Word (.docx), opens in Microsoft Word, Google Docs and Pages. Not legal advice.
Who owns AI agent governance at a company of 100 to 2,000 people
You do not need an AI committee for your first agents. You need four named people per agent, and one person who keeps the company-wide register, usually the COO, CFO or head of IT.
| Role | Decides | Usually |
|---|---|---|
| Agent owner | Signs off go-live, reads the monthly report, can switch the agent off | Head of the team whose work it is |
| Business approver | The job the agent does and its limits | Same team, one level down or up |
| Technical owner | Access, logging, testing and changes | IT lead or the vendor's named engineer |
| Security and data reviewer | Data handling and access before go-live and after big changes | IT security, or an outside reviewer |
One person can hold two roles in a small team, with one exception: the business approver and the technical owner should be different people, so the person setting the limits is not the person building around them.
Testing before go-live: use your own past cases
Vendor demos run on clean examples. Your work is not clean. Before an agent goes live, run it against real past cases from your own records, including ones it should refuse or hand to a person, and agree the pass mark before testing starts. Add four specific tests:
- Injection test: documents and emails with hidden instructions, to confirm the agent ignores them.
- Permission test: the agent attempts an action it is not allowed, and the target system blocks it.
- Limit test: cases just over each limit, to confirm they go to a person.
- Switch-off test: the owner stops the agent and confirms nothing runs after.
This is also the cheapest way to settle whether an agent is worth building at all. Our guide to scoping an AI proof of concept that reaches production covers how to pick those cases.
Monitoring: five numbers with alert levels
| Measure | Why it matters | Alert when |
|---|---|---|
| Share of cases handed to a person | Rising handoffs mean the agent is meeting work it was not built for | Above a daily level you set |
| Approver edit rate | How often a person changes the agent's draft before approving | Above a weekly level you set |
| Failed tool calls | A system changed, a token expired, or an API is down | More than a set number per hour |
| Actions blocked by the target system | The agent tried something outside its permissions | Any |
| Complaints linked to the agent | The signal customers and staff give you for free | Any |
Behavior also changes when the model underneath changes, even if nobody touches your agent. Providers publish retirement dates for their models, so the technical owner should track them and re-run the test set before any switch. Our page on AI agent monitoring and support shows how we handle drift and model changes on agents we run.
A 30-day plan to put governance in place with your first agent
- Week 1, decide. Name the four roles, add the agent to the register, set its autonomy level and fill in the limits table.
- Week 2, restrict. Create the service account, grant only the listed permissions, move keys to a secrets manager, turn on logging for every tool call and approval.
- Week 3, test. Run the agent on your own past cases plus the four specific tests. Fix, re-run, file the results.
- Week 4, start supervised. Sign off, go live at the agreed level, review a daily sample, set the alert levels, and book the first monthly review.
Gartner's 25 June 2025 prediction that over 40% of agentic AI projects will be canceled by the end of 2027 names three causes: escalating costs, unclear business value and inadequate risk controls. A month of governance work done alongside the build deals with the third one directly, and forces an honest answer on the second. For the full picture, see what Gartner, MIT and RAND found about AI project failure.
Governance platforms vs a written policy
Searches for "ai agent governance platform" carry one of the highest click prices in this topic, about $73 in the US (DataForSEO, 17 October 2026), which tells you vendors are competing hard for this buyer. Platforms collect logs, show dashboards and send alerts, and they earn their keep once you run many agents. They do not decide who owns an agent, what it may touch or when a person must approve. Write those down first, then choose tools that record what the policy asks for.
What else ranks for "ai agent governance", and how this guide differs
These are the business sites on Google's first page for "ai agent governance" in the US when we checked on 17 October 2026 (DataForSEO), with the title Google showed for each. We list them so you can compare; we have not reviewed their services, and the list leaves out forums and video sites.
| Site | Page title shown in Google |
|---|---|
| mckinsey.com | Agentic AI security: Risks & governance for enterprises |
| acceldata.io | How Governance-Aware AI Agents Enforce Data Policies |
| indatalabs.com | AI Agent Governance: How to Manage Agents at Scale |
| covasant.com | What Is Agent Governance? The Three Questions |
| opsinsecurity.com | AI Governance and Security: Keeping Agents on Task |
| assistents.ai | AI Agent Governance |
Most of these explain what AI agent governance is. This guide is written for a company of 100 to 2,000 people and comes with a fill-in policy template, a four-level autonomy scale and written approval limits you can adopt this month.
How FactoryJet handles governance on agents we build
We design, build, implement and support custom AI agents, and you own what we build: the code and repository from day one. AI and hosting accounts can run in your name, at cost. We agree the autonomy level, limits and test cases with your agent owner before the build, and we show working software on your own data before you sign. FactoryJet quotes the build, and any support you choose, as fixed prices in writing after a short scoping call.
For one example, read about the accident detection agent we built and still run for Washington Law Group. To see the work by system, start with AI agent development, AI agents for ERP systems or AI consulting. For the bigger choice of what to rent and what to build, read our build versus buy guide for AI agents, and for how the pieces connect, our guide to agentic AI architecture for mid-sized companies.
Sources
All pages opened and read on 17 October 2026.
- NIST, AI Risk Management Framework (AI RMF 1.0 released 26 January 2023; Generative AI Profile NIST-AI-600-1, 26 July 2024)
- OWASP GenAI Security Project, LLM06:2025 Excessive Agency
- OWASP Agentic Security Initiative, Agentic AI: Threats and Mitigations (17 February 2025)
- ISO/IEC 42001:2023, Artificial intelligence: Management system
- Gartner, Gartner Predicts Over 40% of Agentic AI Projects Will Be Canceled by End of 2027 (25 June 2025)
- Our own measurement: US monthly search volumes, year-on-year change and click prices, DataForSEO, 17 October 2026
AI agent scoping call for mid-sized US companies
Get your first agent's limits, logs and autonomy level right before it is built
Tell us the job, the systems and who signs off. We will tell you which level it should start at, what to log and what a person should always approve, and show it working on your own data before you sign. FactoryJet quotes a fixed price in writing after a short scoping call.
Frequently Asked Questions
What is AI agent governance?
What is agentic AI governance?
What are the biggest risks of agentic AI for a business?
Is agentic AI a threat?
What does human in the loop mean for AI agents?
What is the difference between human-in-the-loop and human-on-the-loop?
Is human-in-the-loop still agentic AI?
What should an AI governance policy include?
Is there an AI governance policy template I can use?
What are the four functions of the NIST AI Risk Management Framework?
Do I need ISO 42001 to use AI agents?
What are the governance principles of agentic AI?
What is a major governance challenge for agentic AI?
How do you secure agentic AI?
Who should own AI agent governance in a mid-sized company?
How long does it take to put AI agent governance in place?
Do I need an AI agent governance platform?
Can an AI agent be tricked into doing something it should not?
What should an AI agent never be allowed to do?
How do you monitor an AI agent in production?
What happens when the AI model behind an agent is updated or retired?
Does governance slow AI agent projects down?
What should I ask a vendor about governance before hiring them to build an agent?
What is an AI agent governance framework?
Is there an AI agent governance certification?
Bhavesh Barot
Founder & CEO
Founder & CEO of FactoryJet, an ecommerce and AI services company that has served 500+ businesses since 2014. Writes about ecommerce builds, AI agents, and AI search.
Related reading
- AI Agent Security for Mid-Sized Companies: The OWASP Top 10 Risks and the Control for Each (2026)The OWASP Top 10 for Agentic Applications 2026 explained in plain English for a mid-sized company, with the control for each risk, how prompt injection works and how to limit it, MCP server security, a seven-control checklist, and the security questions to ask any AI agent vendor.
- Agentic AI Architecture for Mid-Sized Companies: A Reference Diagram and the Choices Behind It (2026)A five-layer reference architecture for agentic AI at a mid-sized company: where work arrives, the agent, the tool layer where permissions live, your ERP and CRM, and the control layer. With the five workflow patterns, when to go multi-agent, how MCP fits, and a worked example from emailed purchase order to draft sales order.
- AI Agents for Customer Service: What They Resolve, What They Hand to a Person (2026)How mid-sized companies should use AI agents for customer service: which ticket types an agent can resolve and which go to a person, the systems each needs, rent or build, a good handoff, how to measure it, and Gartner's forecast for agentic AI in service.
