HomeResources

A finding is a claim. A proof is four checks that all pass.

Every Nullgap product follows the same standard. Here is exactly what “proven” means, where the proof runs, what gets ruled out, and how a fix earns “proven fixed”.

FoundProvenFixed✓ Proven fixed
A sealed sandbox with two checked items, beside four labels: Attack, Repeat, Control, Check
The proof standard

Four checks. All of them, every time.

A lead becomes a finding only when every check passes. A status code alone never counts: a “200 OK” says the server answered, not that anything leaked.

  1. Worked1. Attack run

    The exploit is run against your code in the sandbox, and the result shows the flaw: data it shouldn’t return, an action it shouldn’t allow.

  2. Worked2. Repeat

    The same attack runs again, fresh. A one-off fluke doesn’t count.

  3. No effect3. Control run

    A harmless, ordinary input goes through the same path and behaves safely. This shows the effect comes from the attack, not from the app being odd.

  4. Accepted4. Independent check

    A separate reviewer, kept apart from the one that found the lead, looks at the evidence and must agree it shows the flaw.

The whole path

From a lead to proven fixed.

How a lead becomes a proven, fixed finding A lead goes into the sealed sandbox. If all four checks pass it becomes a proven finding; otherwise it is ruled out and still shown. A proven finding gets a fix pull request, the original exploit runs again against the fix on a fresh checkout, and when it fails the finding is proven fixed. Found: a lead Sealed sandbox Proven finding Ruled out, shown Fix pull request Re-test against the fix ✓ Proven fixed four checks, sealed a check fails exploit fails nothing merged for you so you know what was covered closed only when the exploit fails
Where proofs run

A sealed sandbox, built from your code.

Proofs run in an isolated environment with no outside network. Nothing in a proof can reach the internet, your production systems or anyone else.

Built per run
Your repository is checked out for the run and deleted when the run ends. Your source code is not kept.
No outside network
Exploits run against your code inside the sandbox, never against the internet.
Secrets stay masked
Credentials found in your code are masked before storage and never sent to the AI.
Live testing is scoped
When you test a running app, every request is checked against your scope list, private addresses are never reached, and tests are read-only unless you allow state changes.
What a proof looks like

The evidence ships with the finding.

Each finding carries the exact requests and the sandbox output for every run. Anyone on your team can replay it and see the same thing.

What happened
RunWhat it showedResult
Attack runCustomer A fetched customer B’s order exportWorked
RepeatA second, fresh run gave the same resultWorked
Control runCustomer A fetched their own export: normalNo effect
Independent checkA separate reviewer agreed it shows the flawAccepted
After the fix: the same request on a fresh checkout returns 404. Fixed, and proven fixed.
The details that matter

Replay, re-test, and what “ruled out” means.

Anyone can replay

The requests and output for every run are attached. Replay a proof yourself instead of taking our word for it.

Re-tested against the fix

A fix is tested on a fresh checkout and the original exploit runs again. It has to fail before the finding is closed as proven fixed.

Ruled out, in the open

Leads that were tested and couldn’t be proven are listed, so you see what was covered. Ruled out means Nullgap tried and the checks didn’t pass; it isn’t a promise that nothing is there.

Proof across accounts

In live testing, the access matrix shows which of your test accounts could read or change another’s data, each cell proven the same way.

A human when you want one

Ask a Nullgap security researcher to review a finding or a whole report by hand. Their notes are signed “Nullgap researcher”.

Nothing stored that shouldn’t be

AI prompts and responses are never stored. If zero data retention can’t be confirmed, the run stops.

Questions about proof

Why isn’t a status code enough?

A status code says the server answered. It doesn’t say whether anything leaked or changed. Nullgap looks at what came back and compares it with a harmless control run.

What does the control run prove?

That the effect comes from the attack. If an ordinary input produces the same result, the “attack” didn’t show a flaw, and the lead is ruled out.

Who does the independent check?

A separate reviewer in Nullgap AI that is kept apart from the one that found the lead. It sees the evidence and must accept it. You can add expert review by a Nullgap researcher on top.

Does “ruled out” mean we’re safe?

It means Nullgap tested that lead and couldn’t prove it. It shows coverage, not a guarantee. That’s why ruled-out leads are listed rather than hidden.

Do you ever attack production?

No. Proofs run in a sealed sandbox built from your code. Live testing reaches only staging addresses on your scope list, read-only unless you allow state changes.

✓ Proven fixed

Get your first proof today.

Connect a repository and watch Nullgap find, prove and fix. You only ever see what it could prove.

Free 14-day trial · No card · or book a demo with a security engineer