HomeResources
A finding is a claim. A proof is four checks that all pass.
Every Nullgap product follows the same standard. Here is exactly what “proven” means, where the proof runs, what gets ruled out, and how a fix earns “proven fixed”.
Four checks. All of them, every time.
A lead becomes a finding only when every check passes. A status code alone never counts: a “200 OK” says the server answered, not that anything leaked.
- Worked1. Attack run
The exploit is run against your code in the sandbox, and the result shows the flaw: data it shouldn’t return, an action it shouldn’t allow.
- Worked2. Repeat
The same attack runs again, fresh. A one-off fluke doesn’t count.
- No effect3. Control run
A harmless, ordinary input goes through the same path and behaves safely. This shows the effect comes from the attack, not from the app being odd.
- Accepted4. Independent check
A separate reviewer, kept apart from the one that found the lead, looks at the evidence and must agree it shows the flaw.
From a lead to proven fixed.
A sealed sandbox, built from your code.
Proofs run in an isolated environment with no outside network. Nothing in a proof can reach the internet, your production systems or anyone else.
- Built per run
- Your repository is checked out for the run and deleted when the run ends. Your source code is not kept.
- No outside network
- Exploits run against your code inside the sandbox, never against the internet.
- Secrets stay masked
- Credentials found in your code are masked before storage and never sent to the AI.
- Live testing is scoped
- When you test a running app, every request is checked against your scope list, private addresses are never reached, and tests are read-only unless you allow state changes.
The evidence ships with the finding.
Each finding carries the exact requests and the sandbox output for every run. Anyone on your team can replay it and see the same thing.
| Run | What it showed | Result |
|---|---|---|
| Attack run | Customer A fetched customer B’s order export | Worked |
| Repeat | A second, fresh run gave the same result | Worked |
| Control run | Customer A fetched their own export: normal | No effect |
| Independent check | A separate reviewer agreed it shows the flaw | Accepted |
Replay, re-test, and what “ruled out” means.
Anyone can replay
The requests and output for every run are attached. Replay a proof yourself instead of taking our word for it.
Re-tested against the fix
A fix is tested on a fresh checkout and the original exploit runs again. It has to fail before the finding is closed as proven fixed.
Ruled out, in the open
Leads that were tested and couldn’t be proven are listed, so you see what was covered. Ruled out means Nullgap tried and the checks didn’t pass; it isn’t a promise that nothing is there.
Proof across accounts
In live testing, the access matrix shows which of your test accounts could read or change another’s data, each cell proven the same way.
A human when you want one
Ask a Nullgap security researcher to review a finding or a whole report by hand. Their notes are signed “Nullgap researcher”.
Nothing stored that shouldn’t be
AI prompts and responses are never stored. If zero data retention can’t be confirmed, the run stops.
Questions about proof
Why isn’t a status code enough?
A status code says the server answered. It doesn’t say whether anything leaked or changed. Nullgap looks at what came back and compares it with a harmless control run.
What does the control run prove?
That the effect comes from the attack. If an ordinary input produces the same result, the “attack” didn’t show a flaw, and the lead is ruled out.
Who does the independent check?
A separate reviewer in Nullgap AI that is kept apart from the one that found the lead. It sees the evidence and must accept it. You can add expert review by a Nullgap researcher on top.
Does “ruled out” mean we’re safe?
It means Nullgap tested that lead and couldn’t prove it. It shows coverage, not a guarantee. That’s why ruled-out leads are listed rather than hidden.
Do you ever attack production?
No. Proofs run in a sealed sandbox built from your code. Live testing reaches only staging addresses on your scope list, read-only unless you allow state changes.
Get your first proof today.
Connect a repository and watch Nullgap find, prove and fix. You only ever see what it could prove.
Free 14-day trial · No card · or book a demo with a security engineer