What is an AI governance framework and how do you build one?

What an AI governance framework is

An AI governance framework is the set of decisions, roles, and processes that controls how AI is selected, approved, used, and reviewed across the organization. It exists to answer practical questions: What tools are in use? Who approved them? What data may they touch? Who is accountable when an output causes harm?

Public reference frameworks — the NIST AI Risk Management Framework, ISO/IEC 42001, the EU AI Act's requirements — describe what good looks like at scale. They are references, not implementations. Your framework is the subset of those ideas, translated into the decisions your organization actually makes.

The five components every framework needs

1. Inventory

A maintained list of AI tools and AI-enabled features in use, the business process each supports, the data each touches, the owner, and the last review date. This is the foundation; without it, every other component operates on assumptions.

2. Roles and decision rights

One accountable executive, plus defined decision rights for legal, security, data owners, HR, and the business units using AI. The framework should state who can approve a new use case, who decides whether a dataset may be used, and who is accountable for outputs that reach customers or affect employees.

3. Acceptable-use standard

A plain-language document stating what data may never be entered into external tools, which uses are pre-approved, which require review, and the standing rule that a person remains accountable for any output they act on.

4. Risk-tiered review

A proportionate approval path. Low-impact uses (drafting internal summaries) move fast. High-impact uses — anything affecting a person's employment, credit, care, or legal position, or touching regulated data — get structured review before deployment and periodic review after.

5. Monitoring and change

Vendors change models and terms without notice. The framework needs a periodic re-review cycle, an incident route for AI-related issues, and a defined way to retire a tool.

How to build one

• Start with the inventory. Survey teams without blame, review vendor release notes for AI features added to products you already own, and record what you find.
• Assign the accountable owner — usually the executive who already owns technology or information risk — before writing policy.
• Write the acceptable-use standard first. It changes the most behavior for the least effort.
• Define two or three risk tiers with concrete examples, not abstract severity labels.
• Set an approval service level. Slow governance is bypassed governance.
• Schedule the first re-review before the framework launches, so monitoring is built in rather than added later.

What not to do

• Do not adopt NIST, ISO 42001, or a consultant's template wholesale without mapping it to your roles and decisions.
• Do not build the framework around procurement alone — most AI enters through free tools and features inside existing products.
• Do not create a committee as the decision-maker without naming who decides when the committee disagrees.
• Do not publish the framework and stop. Without a review cycle and an inventory that stays current, it expires quietly.

Questions to ask

• If an employee wants to use a new AI tool tomorrow, what exactly do they do, and how long does it take?
• Do we know every AI-enabled feature currently active in tools we already license?
• Which uses could affect a person's employment, care, credit, or legal position — and what review applies to those?
• What contractual commitments have we made to clients about AI use, and does our framework honor them?
• When was our inventory last accurate, and who keeps it that way?

Warning signs

• The framework exists as a document but the inventory does not exist as a practice.
• Approvals happen by email with no record.
• Teams describe governance as something to work around.
• The framework was written before anyone asked what tools were already in use.

When internal teams can handle it

Organizations with functioning risk, security, or technology governance can usually extend those structures to AI. Extending a working structure beats building a parallel one.

When outside help may be appropriate

When the framework must be stood up quickly across functions that do not currently coordinate, when regulatory or client contractual exposure is involved, or when a previous framework was published and never adopted.