LearnNewsExamplesServices
Frontmatter
id17469
titleRepair merged revision-config diff false-success seams
stateClosed
labels
bugaitestingarchitectureagent-ostech-debt
assigneesneo-gpt-emmy
createdAtAug 21, 2026, 3:43 PM
updatedAtAug 21, 2026, 6:10 PM
githubUrlhttps://github.com/neomjs/neo/issues/17469
authorneo-gpt-emmy
commentsCount0
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtAug 21, 2026, 6:10 PM

Repair merged revision-config diff false-success seams

Closed Backlog/active-chunk-18 bugaitestingarchitectureagent-ostech-debt
neo-gpt-emmy
neo-gpt-emmy commented on Aug 21, 2026, 3:43 PM

Context

PR #17459 merged as 338a063e53ca68feef602abf503044d2ef9f3a49, closing #16765 while an operator-triggered debt audit was still running. The trigger was concrete: revisionConfigDiff.mjs exported a native Error subclass that was neither a Neo class nor a useful structured error contract. Regenerating docs after removing it deleted the bogus Neo.ai.scripts.setup.revisionConfigDiff: null class-hierarchy node and changed no other hierarchy entry.

The merged head was green and cross-family approved, but independent exact-source falsifiers found several false-success paths beneath that green. This is a successor, not a reopening: #16765 remains correctly closed by the capability PR; this ticket repairs defects in what shipped.

The Problem

The merged revision differ can emit a valid-looking revision-config-diff.v1 receipt while losing evidence:

  1. fingerprintDefault() catches the same nominal error used for unsupported syntax, missing git objects, parse failures, and cycles. A missing imported object can therefore become a syntax-only “unchanged” default instead of an error.
  2. For 61 of 555 current leaves, default comparison falls back to the leaf-local AST. Changing a referenced helper/constant beneath an unchanged leaf(NAME, …) can disappear; a statically resolvable binding change can instead emit the misleading {from: "NAME", to: "NAME"}.
  3. Caller-injected supportedFromRevision and diffLeafSetsFn can bypass the fixed horizon or suppress added/removed leaves while producing an indistinguishable v1 receipt.
  4. Same-path requiredFor and metadata.parse changes are absent. An optional leaf can become prod-required, or a custom decoder can change, with no path/default/env/type delta.
  5. The CLI defaults to ambient cwd, flattens unexpected failures into ordinary diagnostics, and accepts programmatic target shapes that can turn a claim string into character claims.

These are instrument defects: the tool’s output is evidence used before deployment. A false clean receipt is worse than a loud failure.

The Architectural Reality

  • ai/scripts/setup/revisionConfigDiff.mjs is a pure data-plane parser/CLI. It must stay Neo-free under ADR-0019 C1; Neo.setupClass() is not a repair for a native error subclass.
  • Stable error codes belong on ordinary Error instances here. Expected declaration limits and fatal evidence failures must not share a catch class.
  • Only diffRevisionConfig() may mint revision-config-diff.v1. Lower-level loaded-tree helpers may return unversioned data, never an authoritative schema.
  • diffCohortLeafSets() remains the sole added/removed primitive. No caller injection may replace it.
  • ADR-0019 makes metadata.parse the authoritative custom decoder. Current live census: 8 decoder-bound leaves / 6 decoder identities across Tier 1, Memory Core, GitHub Workflow, and GitLab Workflow.
  • ai/scripts/setup/ remains the correct owner. Structure map: six sibling files; this is the existing revision-loader module, not a new subsystem or placement decision.

The Fix

Repair the merged module and its unit fixture on a fresh branch from current origin/dev:

  • replace the nominal error subclass with a module-local coded-error factory;
  • distinguish deliberate static non-evaluability from I/O, parse, cycle, and missing-binding failures;
  • make syntax-only defaults fingerprint their reachable, scope-aware local/imported binding graph, including external import kind and symbol;
  • compare statically resolved defaults by normalized value and carry useful resolved-value evidence;
  • close the authoritative receipt over the fixed 4749eef99e horizon and imported diffCohortLeafSets;
  • normalize same-path requiredFor and report requiredness-only transitions;
  • classify decoder lifecycle changes as DECODER_BOUND (none → decoder), DECODER_UNBOUND (decoder → none), DECODER_REBOUND (decoder A → B), or DECODER_BODY_CHANGED (same decoder, own body changed); fan out per affected leaf and state the body-digest bound as decoder-own-source-text; imports excluded; formatting-sensitive;
  • anchor direct CLI execution to the module’s repository root, validate target axes, keep expected diagnostics one-line, and preserve unexpected stacks;
  • regenerate docs so the non-Neo class-hierarchy entry disappears.

Contract Ledger Matrix

Target Surface Source of Authority Proposed Behavior Fallback Docs Evidence
authoritative receipt constructor #16765 + this successor only diffRevisionConfig() attaches revision-config-diff.v1; fixed horizon and path-set differ are closed dependencies lower-level helpers return no schema JSDoc typedefs synthetic bypass controls + real-SHA CLI receipts
default projection declared leaf(default, …) source and reachable bindings resolved values compare by normalized value; unresolved expressions include scope-aware binding/import fingerprints missing object/binding/parse/cycle → error JSDoc helper change, nested shadow, package-symbol rebind, missing-object RED controls
requiredness transition normalized descriptor requiredFor optional/required or constraint changes produce changed.requiredFor unreadable shape → error JSDoc requiredness-only control + scalar/list/dedup equivalence control
decoder transition ADR-0019 metadata.parse distinct DECODER_BOUND, DECODER_UNBOUND, DECODER_REBOUND, and DECODER_BODY_CHANGED rows per leaf no decoder at either revision → no decoder row JSDoc + digest bound bound, unbound, rebound, alias-body, negative, and cross-surface fan-out controls
error/class surface pure Node utility boundary + generated hierarchy coded ordinary Errors; no exported native subclass and no class-hierarchy node unexpected errors retain stacks JSDoc docs regeneration exact delta
CLI target/working-root contract direct operator entry repository-root default; non-empty strings/null for scalar axes; string-array/null for claims usage/evidence failure → stderr 1, no receipt module JSDoc + help direct subprocess refusal/help arms

Decision Record impact

aligned-with ADR 0019. The repair continues to inspect declarations without executing revision code or resolving ambient environment values. It adds decoder declaration comparison because ADR-0019 makes metadata.parse authoritative.

Acceptance Criteria

  • No native Error subclass or Neo.ai.scripts.setup.revisionConfigDiff class-hierarchy entry remains.
  • Missing relative objects, unknown bindings, parse failures, and binding/export cycles fail loud; none can degrade to an unchanged default.
  • Same-expression helper changes, nested lexical shadowing, and package imported-symbol rebinding produce default-change rows; equivalent resolved expressions do not.
  • Only diffRevisionConfig() can mint revision-config-diff.v1; callers cannot override the supported horizon or added/removed differ.
  • A requiredness-only transition produces only changed.requiredFor; scalar and singleton-list constraints plus duplicate OR rows normalize equivalently.
  • Decoder appearance, disappearance, rebinding, and same-binding body changes produce four named subclasses: DECODER_BOUND, DECODER_UNBOUND, DECODER_REBOUND, and DECODER_BODY_CHANGED. Bound/unbound preserve fail-closed/fail-open directionality; body-change rows state the own-source/imports-excluded/formatting-sensitive bound; every class fans out across each affected leaf/surface.
  • Programmatic target inputs reject invalid scalar/claim shapes instead of classifying them character-by-character.
  • The direct CLI uses the repository root independent of ambient cwd, keeps expected failures concise, and preserves unexpected stacks.
  • Focused Brain unit evidence, retry-bound lint, AiConfig SSOT lint, engine/Brain boundary, JSDoc, parse, fixed-sleep, docs generation, and real added/changed/pre-horizon CLI probes pass.
  • The repair lands through a new PR targeting dev; #16765 and PR #17459 remain historical links, never reopened or rewritten.

Out of Scope

  • Executing revision-owned AiConfig code.
  • Comparing environment-resolved operator values.
  • Changing the supported history horizon.
  • Adding CI policy around the operator tool.
  • Refactoring unrelated pre-existing custom Error subclasses.

Avoided Traps

  • Registering the error with Neo.setupClass(). This is a pure non-entrypoint parser and must remain Neo-free; the right fix is deleting the needless class abstraction.
  • Keeping injectable dependencies “for testability.” A test seam that can mint an authoritative false receipt is a production bypass.
  • Collapsing decoder lifecycle transitions. Binding, unbinding, rebinding, and same-binding body changes have different boot/validation semantics and operator actions; null payload fields are not a substitute for named kinds.
  • Digest overclaim. The body digest covers the decoder’s own source text only. It misses imported-helper changes and detects formatting churn; the receipt must say so.
  • Reopening #16765. A merged close target is immutable history; this successor owns the repair.

Related

Related: #16765 · PR #17459 · #16489 · ADR-0019

Live latest-open sweep: checked the latest 20 open issues plus recent A2A claims at 2026-08-21T13:42:38Z; no equivalent successor found.

Origin Session ID: 0472bc6f-364a-40ff-80a2-905acdcc6583

Retrieval Hint: query_raw_memories("PR 17459 revisionConfigDiff false-success native Error horizon differ requiredFor decoder body digest")

Authored by Emmy (GPT-5.6 Sol Ultra, Codex).

tobiu referenced in commit 5256c52 - "fix(ai): close revision diff false-success seams (#17469) (#17470) on Aug 21, 2026, 6:10 PM
tobiu closed this issue on Aug 21, 2026, 6:10 PM