Evidence-computed readiness is not a compliance certificate

Abstract

Buyers ask whether an AI control plane makes them “compliant.” That question is ill-posed. Compliance is an organizational assertion against a regime, evidenced over time, often by independent assessors, scoped to a system boundary and a review period. We describe a narrower artifact: readiness, computed for one organization from qualifying product and assurance evidence against a versioned industry-solution catalog. Policy packs and framework mappings are inputs. They are never proof. Catalog-approved wording is selected by computed maturity; marketing cannot request a stronger phrase than the evidence supports. The source of truth is lib/industry-solutions/. Installing a pack does not make an environment SOC 2 certified, HIPAA compliant, PCI DSS compliant, or FedRAMP authorized. We specify qualifying-evidence rules, lifecycle transitions, claim boundaries, and operational consequences of holding last quarter’s green.

1. Introduction

Security vendors have a commercial gravity well: the badge. A heatmap that turns green when a YAML file is attached is easy to sell and easy to audit into a finding. AI control planes are especially tempting because they sit in front of models that already frighten questionnaires. If the product can display “HIPAA” next to a gateway, someone will screenshot it into a deal room.

Intertrace refuses that screenshot. The platform is a security kernel plus investigation UX. Production traffic terminates on Fly agentgateway (https://intertrace.fly.dev, v1.3.1) with itr_rt_* keys, ExtAuthz, fail-closed Go enforcement, and authorize plus permits. That architecture can produce evidence: evaluation results, deployment measurements, decision exports, attestations stored as references. It cannot, by existing, complete a customer’s SOC 2, HIPAA, or agency authorization. Conflating those layers is a claim error, not a missing feature.

This note is a research boundary, not a certification methodology. We do not propose a new assurance standard. We propose that a product which computes maturity must encode the difference between a mapped control and a verified observation, and must fail closed when the observation is missing, self-verified, expired, or out of scope.

2. Why the certificate question is ill-posed

A certificate, attestation, or authorization is issued by a party with a defined role, against a defined system, for a defined period. It is not a property of a software library. A gateway in the data path can be in scope for those exercises. Policy packs can help operators express intended controls. Framework mappings can help discover which intended controls relate to which catalog rows. None of those objects is the assessor’s opinion.

Customers still need a product-shaped answer to “how far along is this solution for us?” That answer can be honest if it is computed, org-scoped, time-scoped, and worded from a catalog. It becomes dishonest if a caller can request target maturity, if a pack ID fills every gate, or if public marketing copy uses assured language while the org’s evidence is preview.

3. Catalog as source of truth

Canonical solutions, claims, evidence requirements, linked pack IDs, framework mappings, catalog gaps, and approved wording live in lib/industry-solutions/ (catalog.ts, types.ts, readiness.ts, claim-language.ts, evidence-lifecycle.ts). Versioned interchange schemas live under contracts/industry-solutions/. Docs and marketing must not freelance stronger claims. Public pages use getPublicIndustryClaimCopy, which defaults to preview wording and must not infer org-scoped readiness from an unauthenticated context.

The catalog contains exactly eight solutions: financial_services, healthcare_life_sciences, customer_support, software_engineering, ecommerce_retail, government_public_sector, legal_professional_services, and insurance. Each has protected workflows, threat outcomes, required controls, evaluations, deployment expectations, assurance obligations, linked packs, mappings, and explicit catalog gaps. Gaps remain visible. Generic detectors and linked packs do not establish that every vertical control exists, and do not make all eight solutions production-ready.

3.1 Maturity ladder

• preview — Catalog definition and approved preview wording exist, but one or more validation requirements lack qualifying evidence.
• validated — All product and assurance requirements through the validated gate have verified, current, in-scope evidence. Repeatable product evaluation plus assurance-scope review. Not production validation and not independent assurance.
• production_ready — Validated requirements also pass the catalog’s product deployment gate (customer-representative deployment or reliability evidence).
• assured — Every earlier requirement plus the catalog’s assured assurance requirement. Current gates require scoped attestation evidence, except government and public sector, which requires boundary-specific authorization evidence.

Each claim is evaluated independently. Solution maturity is the lowest computed maturity of its claims. The response separates product gaps from assurance gaps and returns only catalog-approved wording at the computed level. There is no manual maturity override in the domain model, persistence schema, shared contract, or API. A caller cannot request a target maturity. Readiness is recalculated from evidence and evaluation time, so expiry, revocation, or supersession can demote a claim.

4. Inputs that are not proof

4.1 Policy packs

Packs in lib/policy-packs/registry.ts provide deterministic defaults the gateway merges with organization and asset policy. A solution may link one or more packs as enforcement inputs. A pack ID, compliance tag, framework mapping, or configured default does not satisfy a solution evidence requirement; does not prove a control operated effectively in the claimed workflow; does not prove customer-representative reliability or latency; does not establish a legal agreement, assessment, attestation, certification, or government authorization; and does not make the customer environment compliant.

Runtime still needs packs. Fail-closed ExtAuthz, authorize, and ExtMCP should load real bundles, not CMS pages. That operational necessity is exactly why packs are tempting false proof: they are visible, named, and attached. Attachment is configuration. Configuration is not observation.

4.2 Framework mappings

Mappings are typed. relationship is mapping or aligned_controls. Notes in the catalog state, for example, that a PCI DSS mapping does not establish compliance for a cardholder-data environment, and that SOC 2 aligned controls are not an independent attestation. Public copy uses frameworkMappingDisclaimer(): framework references are mappings or aligned-control notes. They are not independent assessments, attestations, certifications, or government authorizations unless separate scoped evidence says so.

OWASP LLM Top 10 and NIST AI RMF are useful vocabularies. They are not Intertrace certificates. A row that says LLM01 maps to inspect plus permits is a discovery aid. It is not evidence that inspect ran on this org’s MCP traffic last Tuesday.

5. Qualifying evidence

Evidence qualifies only when all of the following hold. Status is verified. A verifier identity is present and differs from the submitter. Organization and scope organization match the organization being evaluated. Solution, claim, requirement, and product or assurance track exactly match the catalog relationship. Scope is marked in scope. Observation time is valid and not in the future. If an expiry exists, it is later than evaluation time; evidence expiring exactly at evaluation time is expired. References are non-empty. Kind matches the requirement (evaluation, deployment_validation, attestation, authorization, and so on).

Consequently, pending, revoked, superseded, expired, out-of-scope, self-verified, cross-organization, wrong-relationship, missing-verifier, and malformed evidence cannot satisfy a gate. Missing verifier fails closed. That list is the product. A heatmap that ignores it will be green and false.

5.1 Kinds and tracks

Tracks are product and assurance. Typical gates pair a product evaluation with an assurance-scope review at validated; a deployment or reliability measurement at production_ready; and attestation or boundary-specific authorization at assured. Kinds include control_test, evaluation, evidence_export, deployment_validation, reliability_measurement, contract, attestation, and authorization. Using an evaluation row to satisfy an attestation requirement is a relationship miss. Using a contract PDF to satisfy a latency measurement is a kind miss. Both fail closed.

6. Lifecycle and separation of duties

industry_solution_evidence stores organization-scoped metadata and references, not restricted report bodies. RLS uses org_members. New submissions start as pending. Valid transitions are pending to verified or revoked; verified to revoked or superseded. Revoked and superseded are terminal. Supersession requires a different same-organization record for the same solution, claim, requirement, and track. Catalog relationship, original submitter, scope, provenance, references, and evidence dates are immutable after submission. Status transitions append to industry_solution_evidence_history.

Submitters cannot verify their own evidence. That is encoded in buildEvidenceTransitionPlan, not in a UI hint. Self-attestation of a control test is a conflict of interest even when the test is real. Independent verification can still be internal (a second role) or external (an assessor). The catalog’s assured gate is where independent attestation or authorization evidence is required; it is not implied by a verified evaluation.

Membership SSOT remains org_members. Humans authenticate with Clerk; entitlements live in Convex; telemetry and evidence live in Supabase. Convex must not become an evidence store. Clerk must not become a certificate. Mission Control’s organization_members stays in sync via triggers and is not an alternate place to “mark HIPAA done.”

7. Approved wording

Each claim carries wording for preview, validated, production_ready, and assured. selectApprovedClaimLanguage refuses a requested maturity above computed maturity. It also rejects unsafe phrases (HIPAA-certified, SOC 2-certified, PCI compliant, FedRAMP-authorized) unless the text is exactly the catalog’s assured wording at assured maturity—an exception for the rare case the catalog itself contains a scoped assurance sentence, not a license for marketing to invent badges.

Example, financial services, preview: “Financial workflow controls are available in preview with PCI DSS and SOC 2 control mappings.” That sentence names mappings. It does not say certified. Validated speaks of repeatable evaluations. Production-ready speaks of a customer-representative deployment with retained decision evidence. Assured speaks of current independent assurance evidence for the stated scope and review period. Each step is a different object. Collapsing them into “we’re SOC 2” is the failure mode this machinery exists to prevent.

7.1 Forbidden claims

• Allowed: computed maturity, suite results, evidence freshness, residual gaps, catalog-approved wording at that maturity.
• Forbidden: “SOC 2 certified by installing pack X,” “HIPAA compliant because the gateway exists,” “FedRAMP authorized because a mapping row exists.”
• Forbidden: inferring org readiness on a public marketing page. Preview catalog copy only, unless an authenticated API computed otherwise.
• SSOT: lib/industry-solutions/ for catalog text and computation. Docs, sales decks, and this blog must not freelance stronger claims.

Healthcare preview wording in the catalog is about PHI safeguards in preview with mappings, not “HIPAA certified.” Government assured wording, when earned, is about boundary-specific authorization evidence, not a blanket FedRAMP authorization for every customer. Those sentences are long on purpose. Short badges are how claim language dies. validateProposedClaimLanguage exists so a UI cannot paste a punchier sentence than the catalog.

8. What the kernel actually contributes

Readiness is not a substitute for runtime enforcement, and enforcement is not a substitute for readiness. The kernel still: derives tenancy from itr_rt_* via ExtAuthz and verify-key; fails closed when the Go enforcer is down; authorizes actions at POST /v1/actions/authorize with scoped TTL permits; inspects MCP /mcp (list/call/resources/prompts), A2A /a2a message/send, and Service /service/* with a per-tenant registry; freezes regex growth (ADR 0004) and ForceAI on protected assets. Those controls produce events and can produce evaluation evidence when suites run against them.

They do not produce a certificate. Railway gateway-v2 is not the product path and must not appear as “in-scope production” for a deployment_validation row unless that is actually what was measured. Preloop is not customer UI and not an assessor. Pins (agentgateway v1.3.1) belong in deployment evidence when the claim is about the Fly path; measuring a laptop Node gateway and labeling it production_ready is a scope lie.

9. Operational consequence

Assurance ops in the product are about producing and retaining evidence, not decorating a heatmap. If an evaluation did not run, readiness must drop or stay unknown—not hold last quarter’s green. If evidence expires at evaluation time, it does not qualify. If a verifier leaves and you cannot show a distinct verified_by, it does not qualify. If the org_id on the evidence row is not the org being scored, it does not qualify even if the screenshot looks right.

1. Run domain suites as evidence producers with timestamps, suite versions, and artifact digests. Store references, not PHI or card data, in industry_solution_evidence.
2. Separate product evaluation owners from verifiers. Encode that in the transition API, not in a standup agreement.
3. Recalculate on read or on a scheduled evaluation time. Do not cache maturity in a column that outlives the evidence.
4. Show catalog gaps next to scores. A production_ready claim with unclosed gaps is a catalog bug; a preview score with visible gaps is honest.
5. Keep sales decks on approved wording. If the computed level is preview, the deck is preview. There is no “commercially assured.”

Incidents remain manual. A failed evaluation can create or update a finding with an audit source; it should not auto-open an incident for every gap. Finding status history and finding_evidence are the investigation trail. Readiness is the catalog function over verified rows. Mixing those objects produces a SOC that thinks a maturity chip is a ticket.

Domain evaluation suites and evidence producers in lib/industry-solutions/ are how product_validation rows get created. They are still not certificates. A green suite on synthetic traffic does not satisfy production_ready without customer-representative deployment evidence. A signed pack deployed to Fly does not satisfy assured without attestation or authorization artifacts referenced from industry_solution_evidence. Producers must record observation time, scope, and references. Recalculation at a later evaluation time must be allowed to demote. Caching a maturity chip in Convex entitlements would mix billing flags with assurance; do not.

10. Related work and limitations

NIST AI RMF’s Govern/Map/Measure/Manage is a lifecycle vocabulary we can map; mapping is not a Measure result. FedRAMP, HIPAA, PCI DSS, and SOC 2 have issuers and scopes this product does not impersonate. Continuous control monitoring literature is closer: observations over time, fail closed on stale data. We are stricter than a typical CCM dashboard in forbidding pack-as-proof, and narrower in computing only cataloged industry-solution claims rather than a general GRC graph.

Limitations: the catalog is Intertrace-authored; it is not a public standard. Qualifying rules can reject true but poorly recorded work. Assured still depends on external attestations the product cannot mint. Eight solutions do not cover every vertical. Evaluation suites can be gamed if producers and verifiers collude; separation of duties reduces that, it does not eliminate it. Runtime coverage (which traffic actually hit the Fly path) is a different measurement from suite coverage; both can be evidence, neither is a certificate.

11. Conclusion

Readiness is a function of qualifying evidence against a versioned catalog, returning approved wording at the lowest earned maturity. Policy packs and framework mappings feed enforcement and discovery. They do not close gates. Installing a pack does not confer SOC 2, HIPAA, PCI, or FedRAMP status. The SSOT is lib/industry-solutions/. Marketing, docs, and APIs that say more than the catalog are bugs.

A security kernel should be proud of computed, demotable scores. Green that cannot go red is not assurance. It is packaging.

References (selected)

• docs/industry-solutions-and-readiness.md — claim boundary, maturity ladder, qualifying evidence, lifecycle.
• lib/industry-solutions/catalog.ts — eight canonical solutions, wording, mappings, gaps (SSOT).
• lib/industry-solutions/readiness.ts — computation; no manual override.
• lib/industry-solutions/claim-language.ts — approved wording selection; unsafe-phrase rejection.
• lib/industry-solutions/evidence-lifecycle.ts — pending/verified/revoked/superseded; submitter ≠ verifier.
• lib/marketing/industry-claim-copy.ts — public preview wording; framework mapping disclaimer.
• NIST AI Risk Management Framework 1.0 — lifecycle vocabulary, not a product certificate.
• SOC 2, HIPAA, PCI DSS, FedRAMP — issued by defined parties against defined boundaries; not pack IDs.
• Intertrace ADR 0005 — Preloop not customer UI; authorize remains decision SSOT; packs compile in, they do not bypass.
• Intertrace ADR 0004 — detection quality via eval and ForceAI, not regex growth presented as proof.