What auditors ask about AI agents, and the evidence that answers them

The questionnaire usually arrives as a new tab in a spreadsheet you have answered for years. Row 214 asks you to describe your controls over AI systems with access to customer data. Row 215 asks for an inventory. The security team attaches the AI acceptable-use policy, writes "Yes, see policy" and moves on. Three weeks later the customer's reviewer sends a follow-up: please provide the list of AI agents in production, the approval record for each one's access, and an example of an action your controls blocked. That is where most programs stall.

The same thing happens in a SOC 2 examination once agents are in scope. The auditor does not ask whether you have an AI policy. They ask for the population of changes to agent access during the period and select items from it. If that population lives in chat threads and people's memory, there is nothing to sample.

Why a policy is not an answer

A policy states intent. Evidence shows that a control operated, for a specific item, on a specific date. A SOC 2 Type 2 report makes the distinction explicit: the auditor's opinion covers whether controls were designed appropriately, were in operation on a specified date and were operating effectively over a specified time period [2]. A policy saying agents need approval for sensitive actions is design. Twenty approval records drawn from across the period, each naming the approver and the action, are operation.

ISO/IEC 42001, the AI management system standard, frames the same idea as a system to be established, implemented, maintained and continually improved, and it applies to any organization that provides or uses AI-based products or services [3]. The NIST AI RMF describes outcomes rather than documents, such as mechanisms being in place to inventory AI systems [4]. In each case the reviewer is looking for the mechanism and its output, not a statement that one should exist.

The figure shows why agent programs fail at the first step. A sample only means something if it comes from a complete population, so an auditor who cannot get a list of every agent, or of every access change, cannot test anything downstream. Weak evidence is the second failure: a screenshot of a settings page shows the configuration today, not on the date of the change the auditor picked.

The six questions, and what satisfies each

Questionnaires word them differently, but nearly every AI section comes down to six questions. For each one the reviewer is testing a specific property, and some kinds of evidence satisfy it far better than others.

1. Inventory: the population everything else depends on

The NIST AI RMF asks for mechanisms to inventory AI systems, resourced according to organizational risk priorities [4]. The reviewer's real question is completeness, so an inventory alone is weak and an inventory with a reconciliation is strong. Each month, compare the list with what you can discover independently: AI provider keys in your organization accounts, AI apps granted access in your identity provider, model SDKs and MCP configurations in your repositories. Record each difference and what you did about it. A reconciliation that found two unregistered agents and closed them is better evidence than one that never finds anything.

2. Access: the configuration, its history and the review

For each agent, list the tools and credentials it holds and what each credential can actually do, not what it was meant to do. Then show history: every change during the period, who requested it, who approved it and when. Finally, show a periodic review in which the agent's owner confirmed the access is still needed. Instructions in a system prompt are not an access control and should never be offered as one. An auditor who accepts them as a control will test them, and they will fail.

3. Approvals: the rule and the records

Human oversight is the question with the most room for overclaiming. The rule is design; the records are operation. For a sampled action, the record should show what was requested, including its arguments, who approved it, when, and what happened next. Include the negative case too: an action that needed approval, did not get it and was blocked. MANAGE 2.4 in the NIST AI RMF asks for mechanisms to supersede, disengage or deactivate AI systems whose outcomes are inconsistent with intended use [4]. Approval and block records are the everyday form of that mechanism.

4. Monitoring: coverage, not dashboards

MANAGE 4.1 calls for post-deployment monitoring plans that are actually implemented [4]. A dashboard screenshot shows that a dashboard exists. Coverage shows that monitoring reaches the agents in scope: what share of agent traffic passes through the controls that raise alerts, and which agents sit outside them. Then trace three or four sampled alerts from detection to a closed ticket.

5. Incidents: AI categories in the process you already have

You do not need a separate AI incident process, but the one you have needs AI categories: a prompt injection that changed an agent's behavior, data disclosed through an agent, an agent action nobody authorized, and an agent that could not be stopped. MANAGE 4.3 asks that incidents and errors be communicated to relevant AI actors and that processes for tracking, responding to and recovering from them are followed and documented [4]. A near-miss log is persuasive here because it shows the categories are in use.

6. Vendor AI: the register and the review

GOVERN 6.1 calls for policies and procedures that address AI risks from third-party entities, and MANAGE 3.1 for regular monitoring of third-party AI risks with controls applied and documented [4]. The evidence is a vendor register that flags which vendors run AI on your data or act in your systems, the review record for each AI feature, and the contract terms that came out of it. A vendor agent that acts inside your systems belongs in your agent inventory as well as your vendor register.

A worked example: the follow-up questionnaire

Consider Pelham Ledger, a fictional accounting software company of 600 people heading into its second SOC 2 Type 2 period while answering an enterprise prospect's questionnaire. It runs a support agent with access to the ticketing system and customer account records, an invoice-matching agent with read access to the general ledger, and coding agents used by its engineers.

Its first answers were policies. The prospect asked for the agent inventory and an example approval record; the auditor asked for the population of access changes to agents during the period. Pelham could produce neither from a system. Assembling them by hand took two engineers nine working days (an illustrative figure) and turned up three problems: a marketing team's summarization agent running on a personal API key, a contractor's coding agent holding a repository token with write access to the release branch, and two changes to the support agent's tools with no recorded approver.

Over the next eight weeks Pelham made its controls produce their own evidence. Agents were registered with owners, and their calls went through a single control point that recorded every decision. Access changes went through a reviewed change process, so the history became a query. A monthly reconciliation compared the inventory with discovery of keys and grants. When fieldwork came, the population was an export and the sample was drawn in an afternoon. The one exception found, an approval granted by someone outside the approver group, was a real finding rather than a missing record, and it was fixed with a rule change.

Build the evidence as a by-product, not a project

The difference between Pelham's two attempts is where the evidence came from. Evidence assembled for the audit is expensive, incomplete and easy to doubt. Evidence produced by the control as it operates is cheap, complete and dated.

• Record at the decision point. Each agent action should produce one record with who acted, what was attempted, the decision, the rule that made it and the approver, if any. A record written by the control is stronger evidence than a log line the agent wrote about itself.
• Keep records append-only and retained past the end of the audit period. A Type 2 report covers a period of time [2], and a sampled item can come from any day in it.
• Save the queries. The population of access changes, the approval sample and the monitoring coverage figure should each be a saved, versioned query, so next year's evidence is produced the same way.
• Schedule the reviews. A monthly inventory reconciliation and a quarterly access review, each with a sign-off record, turn two of the six questions into routine output.
• Record exceptions with an owner and an expiry date. Auditors accept a documented, time-limited exception far more readily than an undocumented permanent one.

Map to frameworks without overclaiming

SOC 2 reports against the AICPA Trust Services Criteria, which cover security, availability, processing integrity, confidentiality and privacy [1]. Agents do not need new criteria. They are systems and non-human users that existing criteria already reach, such as logical access, change management, monitoring and vendor management. ISO/IEC 42001 adds a management system for AI [3], and the NIST AI RMF organizes outcomes into four functions: govern, map, measure and manage [4]. One body of evidence can serve all three if each record is mapped to the controls it supports.

Be careful with language. A mapping table shows reviewers where to look; it is evidence for their review, not an attestation. Only a CPA firm's opinion attests to SOC 2 controls, and only an accredited certification body certifies an ISO/IEC 42001 management system. Phrases such as "aligned with" or "42001-ready" invite the follow-up question of who said so, and the honest answer is usually that you did.

What goes wrong in practice

• The inventory covers agents the company built and misses the ones it bought or that contractors brought. Vendor and contractor agents act with your credentials and belong in the population.
• Approval evidence shows that something was approved but not what. Without the arguments, a reviewer cannot tell whether the person approved the action that actually ran.
• Logs exist but expire before fieldwork. Check retention against the end of the audit period plus the time the auditor needs to sample.
• Different people answer the same questionnaire question differently each time. Keep one approved answer per question, with its evidence link, and update it when the control changes.

Where a runtime gateway helps

Intertrace sits inline between agents and the models, tools and MCP servers they call, and records every decision as a structured event with who acted, what was attempted, the decision, the rule and layer that made it, and the evidence behind it; events can be exported to your SIEM for retention. Approvals land in one inbox and are recorded with the approver, and exceptions are named, scoped to one agent and expire. Read-only discovery of cloud accounts, repositories, identity-provider AI-app grants and AI provider keys supports the monthly inventory reconciliation. Framework mappings and compliance pack exports are evidence for your auditor to review; they are not an attestation and do not make you compliant with any framework.

What to do this quarter

1. Build the population. List every agent, including vendor and contractor agents, with an owner, purpose, model, credentials and data classes, then reconcile the list against discovery of keys, grants and repository usage.
2. Write one approved answer to each of the six questions, each pointing to the evidence that supports it rather than to a policy.
3. Move agent access changes into a reviewed change process, so the history names a requester and an approver for every change.
4. Turn approval and block decisions into retained records that include the action's arguments, and confirm retention covers the audit period.
5. Save the queries for the access-change population, an approval sample and monitoring coverage, and run them once as a dry run with a colleague playing the auditor.
6. Add AI categories to your incident process and log the first near miss.
7. Flag AI vendors and vendor agents in the vendor register and attach the review record for each.