AI governance: who owns the outcome?

A small team examines plans and a building model beside a red stop button in a warmly lit workspace.

Build with AI · Sources reviewed 2026-11-19

AI governance is the working agreement behind an AI system: what it is meant to do, who owns the outcome, what evidence supports its use, and what happens when the result is wrong.

A small business can express that agreement in a concise operating record. An enterprise may need several teams and formal review. Both need a decision path that remains usable after the demonstration is over.

This guide follows a fictional assistant that drafts descriptions for an adult-learning course catalog. It uses approved course outlines and produces drafts for an editor. The example is a proposed operating design, not a reported deployment.

On this page
Information owners and system operators provide evidence to reviewers. Reviewers and incident reports inform the business owner, who decides whether to continue, change, or pause the service.
Roles are connected by concrete evidence and decisions. One person may hold several roles, but the source, review, operating, and escalation responsibilities still need named owners. Diagram: Root Access Guy.

Start with the use case and the people affected

Write the task in terms of the work: “Prepare a course-description draft that accurately reflects the current approved outline.” Identify the instructors, editors, students, and support staff who may be affected by an incorrect description.

Record the inputs, outputs, actions, and limits. In this example, the assistant may read approved outlines and save a draft. It must leave absent prerequisites, accessibility arrangements, and course guarantees unresolved. An editor checks those items before publication.

The NIST AI Risk Management Framework is voluntary guidance for incorporating trustworthiness into AI design, development, use, and evaluation. Its Core organizes activities through Govern, Map, Measure, and Manage. Use those functions as a structure for decisions about this deployment; they do not certify an individual output or replace applicable obligations.

Name owners who can make decisions

Role Practical responsibility Evidence they need
Business owner Decide whether the assistant is useful and appropriate Quality, user impact, workload, and cost
System operator Maintain the application and handle failures Runs, errors, versions, access, and service health
Information owner Approve sources and resolve conflicting content Source changes, effective dates, and correction reports
Reviewer Accept, edit, or reject a particular draft Original outline, draft, omissions, and differences
Escalation owner Resolve issues beyond the reviewer Severity, affected records, and available containment actions

One person can hold several roles, but record them explicitly. A name on a document is useful only when that person can see the evidence and has authority to act.

For the catalog, Daniel owns the accuracy of course promises. The publishing administrator operates the connection. Instructors maintain source outlines. Editors review drafts. Daniel can pause the use case when the evidence calls its suitability into question.

Create a short operating record

Use an entry like this as a starting point:

System: Course-description drafting assistant
Purpose: Produce reviewable descriptions from approved course outlines
Business owner: Catalog manager
Operator: Publishing administrator
Sources: Approved outline collection; current effective versions only
Permitted output: Draft description with source references and open questions
External action: Publication occurs through the editor's normal workflow
Known limitation: Source omissions cannot be resolved from model fluency
Review evidence: Outline, proposed description, unsupported claims, changes
Pause signal: Any invented course promise reaching a published listing
Change review: Model, prompt, connector, permission, or source-policy changes
Correction route: Editor reports record ID to catalog manager

Replace the role labels with actual accountable people when deploying. Store the record where operators and reviewers can find it. Give it a version and an owner rather than leaving it in the original project chat.

Turn quality goals into observable checks

“Accurate and helpful” needs a more concrete interpretation. For this example, define checks such as preserving course duration, correctly identifying prerequisites, citing the current outline, and explicitly identifying absent accessibility information.

Build a pilot set with ordinary outlines, incomplete sources, conflicting versions, unusual course formats, and text containing unrelated instructions. Have subject-matter reviewers establish the expected outcomes before inspecting model performance.

Choose release criteria according to the consequence of an error. As an illustrative pilot rule, require every seeded contradiction to be flagged and every draft to retain its outline reference. Treat that rule as a gate on the test set, not a guarantee about all future inputs. Record unsupported claims, reviewer corrections, review time, and cases where the assistant appropriately declined to finish.

Anthropic’s evaluation guidance explains how repeated trials and explicit grading help reveal variation in agent behavior. Keep a stable set of important cases so a change can be compared with the previous version.

Match access and review to the actual consequence

An assistant drafting internal text needs a different operating boundary from a system changing customer records or making recommendations about people. Identify the actual action and affected information before deciding the review process.

Keep the catalog source collection separate from private enrollment or payment data. Give the application only the operations required to prepare drafts. Enforce permission checks in the destination system and application code. OWASP’s excessive-agency guidance explains how unnecessary functions, privileges, or autonomy increase the potential impact of a model error.

Review needs to be meaningful. An editor should see the source beside the proposed wording and be able to change or reject it. Record what was approved. If the text changes afterward, the original decision does not cover the new version automatically.

Prepare for a correction before one is needed

Suppose a draft promises an exam certification that the course outline never included. Preserve the source version, generated draft, system version, and editor action. Correct affected published material through the normal publishing process, and investigate whether the problem arose from the source, instructions, model behavior, or review design.

Use the finding to update the evaluation set. Recheck related drafts if the failure could recur. Resume the affected capability only after the responsible owner accepts the evidence for the change.

Keep an accessible route for a student, instructor, or staff member to report a concern. An internal run log is useful to engineers; a person affected by an error also needs a clear contact and an explanation of the correction process.

Operate and change the system deliberately

Track accepted drafts, correction categories, exceptions, source freshness, cost, and connection failures. Review the signals at a frequency suited to volume and consequence. A low-volume editorial assistant may need a different cadence from a customer-facing service.

Record changes to prompts, model versions, retrieval settings, source collections, and permissions. Test material changes with the same representative cases before broad use. Include a rollback or manual-work option so a failure does not leave the team unable to finish essential work.

Budget for review and maintenance as well as software. A tool can lower drafting effort while increasing verification effort. Compare the complete task and the quality of its result.

For regulated or consequential applications, involve the appropriate domain, privacy, security, and legal owners. Start with the specific jurisdiction, data, and use case. This guide provides an operating design, not a finding that a particular system meets every obligation.

Continue with workflow design, then apply the same ownership record to the healthcare or legal-services context.

Keep building

Next guide: AI in every industry

Explore Build with AI

Share this page