When a category gets hot, the demos get good faster than the products do. AI security is there now: polished dashboards, confident language, and — underneath — a wide gap between tools that observe your AI and tools that can actually stop something in real time. You don't need to be an engineer to tell them apart. You need to ask a few blunt questions and listen for whether the answer is a capability or a hope. Here are the ten we'd ask.
Does it watch, or does it enforce?
This is the first fork and it changes everything downstream. A monitoring tool reads logs and tells you after the fact; an enforcement tool sits in the path of your AI traffic and can block, redact, or deny in the moment. Monitoring is useful and cheaper to adopt, but it cannot prevent a leak — it can only describe one. If a vendor blurs this line, make them say plainly which one they are.
- When something bad happens, do you stop it or just tell me about it? Enforcement prevents; monitoring reports. Know which you're buying.
- What happens when your product fails or is overloaded — does my AI traffic still get protected, or does it sail through? 'Fail open' means the guard disappears exactly when it's stressed. Insist on fail-closed.
- Can you see what my AI is about to send out, not just what comes in? Most data leaks happen on the way out. Input-only inspection misses them.
- How do you keep one customer's data — or one team's agent — from touching another's? Ask specifically how tenants and agents are isolated.
- Can you show me every AI tool and agent my company is actually using, including the ones nobody registered? Discovery is the step most tools skip and most breaches exploit.
Can it prove what it did?
The second half of the list is about evidence and honesty. A control that can't produce a record of its decisions is a control you can't defend to an auditor or reconstruct after an incident. And a vendor that promises certainty about AI — 'we stop all prompt injection' — is telling you something the entire security field says isn't true.
- If a regulator or auditor asks what your product blocked last Tuesday, can you produce it? Durable, exportable evidence separates a real control from a black box.
- Do your compliance features claim certification, or do they support my audit? A tool can generate evidence for SOC 2 or HIPAA work; it cannot be your certification. Be suspicious of badge language.
- What do you claim you cannot do? Prompt injection is not 'solved' — the honest vendors say so. A pitch with no limits is a pitch to distrust.
- How much latency do you add, and can I leave it on in production? A control people turn off because it's slow protects nothing.
- How fast can I revoke access or shut an agent down myself? You should be able to pull the plug without waiting on the vendor.
How to read the answers
You're listening for three things: enforcement over observation, fail-closed over fail-open, and evidence over assertion. A vendor who answers those three cleanly — and who tells you honestly what they can't do — is selling a control. A vendor who answers with dashboards, certainty, and badges is selling comfort. Both cost money; only one lowers your risk.