How Much of a Repository Should an AI Reviewer Read?

A developer changes one line in a pull request. That line calls a helper somewhere else in the repository. Should an AI reviewer read the whole project to understand it?

It could. But that would send more private source than the review needs, cost more to process, and still give no guarantee that the model uses the extra code well.

When I designed Scopebase's source collector, I wanted the model to get enough context to reason about a change, without treating the repository as one enormous prompt. The collector builds a bounded input first. If it has to omit relevant code, it carries that fact forward as partial coverage.

Start with the diff, then follow a little context

Scopebase reads immutable Git blobs and selects changed hunks with up to 20 nearby lines. That keeps the review centered on what changed while giving the model local context around each edit.

Sometimes a changed line calls a helper. The collector can resolve up to eight direct local dependencies. It does not recursively wander through the codebase, execute repository scripts, or treat configuration files as instructions.

I like this boundary because it's predictable. The system can explain which changed code and nearby dependencies were selected. A recursive search would be harder to bound and would make “what exactly did the model see?” a much fuzzier question.

Put a ceiling on the input

The recorded policy caps each file at 64 KiB and the complete selected source snapshot at 256 KiB. Selection follows deterministic ordering. Source preparation gets 30 seconds within a 120-second deadline for the complete engine invocation.

Those numbers make the problem concrete. A file can be too large. A dependency can fail to resolve. The total selected input can hit its cap. In each case, the collector has to decide what fits and mark coverage partial when eligible context is left out.

Partial is important. If a helper was omitted, “no findings” means no findings in the source the model received. It cannot honestly mean the reviewer examined the whole relevant behavior.

The collection path is a single sequence. If eligible context is omitted, the coverage status records that before the model call:

Screen the removed code too

The diff contains added and removed lines. A deleted line can still hold a credential. So can a dependency the collector included for context. Screening applies to selected text, not just the new side of the patch.

That reduces exposure, but screening cannot prove that every secret has been found. I don't want the source-selection policy to imply a guarantee that no filter can make.

The source and prompts stay transient in worker memory. Persistent records keep screened findings, safe metadata, and accounting facts rather than source snapshots. That makes reproducing an old model request less convenient, but it also reduces sensitive retention and the amount of private data deletion has to cover.

Prepare the input before spending

Scopebase prepares source and computes a quote before contacting the model. Admission then checks current ownership, eligibility, entitlement, credits, and internal spending capacity in a short database transaction.

This ordering matters for two reasons. The system can reject the review before source is sent to a provider, and the quote is based on the input it actually prepared rather than a guess made before collection.

Source selection is part of the feature's correctness. It decides what the model can see, what leaves the worker, how much the invocation can cost, and whether the result must carry a partial-coverage caveat. It is every bit as much a product decision as the prompt wording.

More context is a hypothesis, not a fix

These limits can omit something useful. A bug might depend on a second-order helper. Secret screening might miss a credential. Even when all the relevant code is present, a model can still misunderstand it.

So I don't think “read more code” should be the default response to a missed bug. First I want evidence about what context the miss needed and whether the current collector could have supplied it. Then the selection rule can change for a reason.

The goal isn't the smallest prompt at any cost. It's a bounded, explainable view of the changed behavior, with omissions visible and private source kept to what the review needs. If evaluation shows that direct dependencies aren't enough, that's a concrete reason to revisit the boundary.