LearnNewsExamplesServices
Frontmatter
id16196
titleRefine believed-open unverifiable reasons
stateOpen
labels
enhancementaitesting
assigneesneo-gpt-emmy
createdAtJul 30, 2026, 11:44 PM
updatedAtJul 30, 2026, 11:55 PM
githubUrlhttps://github.com/neomjs/neo/issues/16196
authorneo-gpt-emmy
commentsCount0
parentIssue16136
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]

Refine believed-open unverifiable reasons

Open Backlog/active-chunk-11 enhancementaitesting
neo-gpt-emmy
neo-gpt-emmy commented on Jul 30, 2026, 11:44 PM

Summary

Use the partial GraphQL evidence already returned by the believed-open query to make unverifiable entries actionable without another GitHub request.

The response contract should distinguish only states the wire actually proves:

  • not-found — a null aliased row plus an exact alias-scoped GitHub NOT_FOUND error.
  • unrecognized-state — an exact-number aliased row exists, has no contradictory alias error, but carries an unknown state.
  • lookup-error — an exact alias-scoped non-NOT_FOUND error or a contradictory row-plus-error payload.
  • unresolved — no usable exact-number row and no exact alias-scoped error.

The existing three-bucket safety partition remains unchanged: no uncertain, malformed, or wrong-number observation may enter falsified.

Problem

GraphqlService.query(..., {strict: false}) already returns {data, errors} for partial responses, but PullRequestService.listPullRequests() keeps response.data and discards response.errors. projectBelievedOpen() therefore emits the same not-found-or-inaccessible reason for every null or unknown row.

A live GitHub GraphQL query falsifies the proposed three-way “not found / inaccessible / unrecognized state” split. An inaccessible repository is a top-level repository failure, not a distinct per-number signal. For a missing pull request in an accessible repository, GitHub returns:

{
  "data": {"repository": {"missing": null}},
  "errors": [{
    "type": "NOT_FOUND",
    "path": ["repository", "missing"],
    "message": "Could not resolve to a PullRequest with the number of 99999999."
  }]
}

The stable evidence is the typed error plus its exact alias path. Human-readable message text and a per-number access diagnosis are not authoritative.

The current classifier also trusts an aliased row's state without proving that row.number matches the submitted coordinate. A malformed or mis-associated terminal row must fail closed rather than falsely refute the caller's belief.

Architectural reality

  • GraphqlService.query() owns strict versus partial GraphQL response handling and already preserves typed errors in non-strict mode.
  • PullRequestService.listPullRequests() owns the one-request board-plus-alias query and currently drops the errors array.
  • projectBelievedOpen() owns the exhaustive, disjoint three-bucket projection.
  • PullRequestBelief.unverifiable[].reason in the GitHub Workflow OpenAPI document owns the agent-consumed reason enum.

No new request, MCP operation, query-builder behavior, or subsystem is required.

Contract Ledger

Target Surface Source of Authority Proposed Behavior Fallback / Edge Case Docs Evidence
Existing PullRequestBelief.unverifiable[].reason Live GitHub typed partial-response evidence plus parent #16136 Emit not-found only for a null row plus an exact ['repository', alias] NOT_FOUND; emit unrecognized-state only for an exact-number row with an unknown state and no alias error Emit lookup-error for exact non-NOT_FOUND errors or contradictory row/error payloads; emit unresolved when neither a usable row nor exact error exists Update OpenAPI enum and descriptions Service projection tests plus emitted MCP schema assertion
Existing PullRequestService.listPullRequests({believedOpen}) partial-response path GraphqlService.query(..., {strict: false}) Pass returned errors alongside repository data into the classifier while retaining one GitHub request Repository-level/no-partial-data failures remain the existing top-level GRAPHQL_API_ERROR path Keep JSDoc parameter/return contracts complete Query-count assertion and repository-level failure regression
Existing belief.stillOpen and belief.falsified buckets Parent #16136 safety contract Classify a state only when the aliased row's numeric number exactly matches the submitted number; only exact CLOSED or MERGED rows enter falsified Missing, malformed, wrong-number, contradictory, unknown, or provider-error rows remain unverifiable No bucket-shape change Exhaustive partition test across open, terminal, missing, wrong-number, unknown, contradictory, and unmapped-error rows

Acceptance Criteria

  • Partial response.errors map to a submitted number only by the exact ['repository', alias] path; extra-depth, reordered, unrelated, missing, and non-array paths do not match.
  • Human-readable GitHub error messages are never parsed, and only exact type: 'NOT_FOUND' qualifies for not-found.
  • A null row plus exact alias-scoped NOT_FOUND produces reason: 'not-found'.
  • An exact-number row with an unrecognized state and no alias error produces reason: 'unrecognized-state'.
  • An exact alias-scoped non-NOT_FOUND error, duplicate/contradictory error set, or row-plus-error contradiction produces reason: 'lookup-error'.
  • A missing, malformed, or wrong-number row with no exact alias error produces reason: 'unresolved'.
  • OPEN, CLOSED, and MERGED classification requires row.number === submittedNumber; a wrong, missing, or non-numeric row number cannot enter stillOpen or falsified.
  • The OpenAPI reason enum and emitted tools/list output schema expose exactly the four supported reasons.
  • The query remains one GitHub request, the no-belief/default path remains unchanged, and repository-level GraphQL failures retain their existing error path.
  • No unobserved row can enter falsified; the submitted coordinate remains exhaustively and disjointly partitioned.

Out of Scope

  • Claiming that GitHub distinguishes inaccessible from nonexistent pull requests per number.
  • Exposing raw provider messages, paths, error types, or payloads to callers.
  • Changing the stillOpen / falsified / unverifiable buckets, request cap, or query count.
  • Changing the query builder or GraphqlService.
  • The OpenAPI array-constraint compiler repair tracked separately in #16194.

Avoided Traps

  • Parsing GitHub's human-readable error message.
  • Treating NOT_FOUND as proof of an access-control state.
  • Inventing a per-number inaccessible reason the wire does not expose.
  • Trusting state without exact submitted-number identity.
  • Sending unknown, malformed, or contradictory rows into falsified.
  • Expanding the already-open contract-repair PR.

Decision Record Impact

None. This refines an existing MCP response enum using evidence already returned by the current request.

Relationships

Parent: #16136

Successor to #16191

Related: #16192, #16194, #16195

Retrieval Hint

believedOpen response.errors alias NOT_FOUND exact row number unverifiable unrecognized-state lookup-error unresolved

Origin Session

b1ebc46a-5a83-496c-aa8b-385af785e9cb