“No Findings” Can Mean Four Different Things

A developer opens a pull request check and sees no findings.

Did the reviewer inspect the change and find no supported bug? Did it miss a dependency because the source limit was reached? Was analysis skipped? Did it fail before finishing?

The same empty list can mean all four things. If the interface turns every one into a green “all clear,” it tells the developer more than the system knows.

When I designed Scopebase's result model, I wanted it to say what happened, including when analysis did not produce a usable answer.

An empty array isn't a result

These are different outcomes, even if each one has zero findings to display:

Only the complete case means the defined review ran with sufficient collected context and returned no supported finding. Even then, it doesn't prove the change is bug-free. It means the engine found nothing that passed its rules.

Partial analysis needs a qualifier. Skipped and failed analysis need an explanation. None of them should look like a clean review just because the findings array happens to be empty.

A finding has to survive several checks

The first model pass proposes up to ten candidate bugs. Each one needs a trigger, a consequence, an anchor on an added line, and evidence ranges in the source it received.

Then a second pass gets the code, diff, and candidate claims in a fresh request. Its job is to challenge reachability, look for counterexamples, and decide whether the change introduced the failure. It can accept or reject candidates. It cannot invent new ones.

After that, local checks validate what the application can verify directly. The anchor must be on the right side of the diff and point to an added line. The evidence must exist in the supplied snippets. Scopebase screens output for source and secret leakage, removes duplicates, and returns at most five findings.

If discovery produces no valid candidates, verification is skipped. If the second pass returns malformed or incomplete verdicts, the candidates are not published.

I like that each stage has a narrow job: the models reason about behavior, and local checks validate structural facts. But two passes do not prove accuracy. The models can share the same blind spot, and local checks cannot prove a behavior claim is true. We still need human-adjudicated evaluation to know whether the findings deserve attention.

Analysis and delivery are separate

Suppose analysis succeeds but GitHub delivery fails. The review result still exists. Running the model again would spend more budget without fixing the delivery problem.

Or delivery may succeed with a check explaining that analysis failed. That is useful information too. The developer can see that the system ran into a problem instead of receiving a misleading empty list.

Scopebase tracks analysis outcome separately from delivery state. It also tracks whether the result is still available. Those are three questions: what did analysis do, what happened when the result was sent, and can the user still read it?

Results expire. History reads enforce expiry even if cleanup is delayed, and the browser clears private results when their conservative display interval ends. An expired result should be represented as unavailable, not as zero findings.

History needs fresh permission checks

Review history contains private repository information. A person who had access when the review started may lose that access later, so a history read checks current authorization again.

That makes the result model more involved than storing a comment array and a status boolean. But the user-facing choice is simple: explain what the system did, whether it reached GitHub, and whether the result is still available to this person.

The UI does not need to expose every internal state name. It does need to keep the distinctions intact so it can say “analysis was partial,” “delivery is still unresolved,” or “this result has expired” when those are the facts.

Report the work, not a promise

An AI reviewer should not tell a team “all clear” after skipping analysis, omitting relevant source, or losing the result. It should not throw away completed analysis because GitHub delivery failed either.

Scopebase's separate outcome, delivery, and availability states add complexity. They also give the product a way to describe uncertainty without turning it into reassurance.

The bigger question is still model quality. Local checks can show that a finding points to code the model received and an added line. They cannot show how often the model finds real bugs or how often it misses them. Until we have that evaluation, the most honest result is an account of what the workflow did, not a certificate that the code is safe.