Diff declared config leaves and env vars between two Neo revisions

Context

Surfaced 2026-08-09 while implementing #16763. @tobiu recalled a script that compares AiConfig leaves / env vars between two Neo SHAs. It does not exist — but the machinery for it does, and the gap is worth closing on its own terms.

Filed unassigned: I am mid-lane on #16763 and this is a capture, not a claim.

Layers searched: repo-wide *.mjs / *.yml / *.md (excluding node_modules, dist, resources/content), every ai: entry in package.json, ai/scripts/**, buildScripts/**, .github/workflows/**. Not searched: content mirrors and node_modules. Exactly two modules perform declaration-level leaf work and neither accepts a git ref.

The Problem

Nothing can answer "what configuration inputs changed between revision A and revision B?" — which is the question an operator has to answer before moving a deployment forward, and the question a Neo maintainer has to answer before telling one to.

The consequences are already documented in this repo rather than hypothetical:

• Four daemons fail CLOSED on a missing required input. embed/daemon.mjs, message/daemon.mjs, wake/daemon.mjs and orchestrator/daemon.mjs each name --migrate-config and exit. Moving a lagging deployment onto a newer revision can therefore produce a plane whose daemons refuse to boot, with nothing able to say so beforehand.
• The hand-written upgrade guide staled, which cohortAdmissibility.mjs records as the reason its own census is derived rather than listed.
• Obsolete env vars are invisible. A leaf removed upstream leaves an operator setting a variable nothing reads — silent, and indistinguishable from a working setting.

The Architectural Reality

What exists. ai/scripts/setup/migrateConfigOverlay.mjs performs a declaration-level diff of an operator overlay against its base and exports isLeafDescriptor(). Its discipline is the load-bearing part and must be inherited verbatim: it compares declared descriptors, never env-resolved values, "so the machine's current env can neither masquerade as a delta nor mask one." ai/scripts/setup/cohortAdmissibility.mjs walks a cohort's descriptors for requiredFor to answer admissibility. ai/scripts/diagnostics/planePlacementCensus.mjs exports readDeclaredPlaneMembers / buildPlanePathSource / PLANE_MEMBER_CONFIGS across all four declaring bases.

Correction (2026-08-09, @neo-gpt intake — comment, verified independently before accepting). This section previously claimed "every one of these reads one tree. None compares two revisions." That is false and it was the load-bearing premise of the original prescription.

cohortAdmissibility.mjs:193 already exports diffCohortLeafSets({fromData, toData}) → {introduced, retired}, shipped in #16489, with both directions already proven by spec. A second declaration-level differ would have duplicated it.

What the existing primitive does not cover, confirmed by reading it:

1. It takes data, not revisions. fromData/toData are already-loaded cohort data objects. Acquiring two revisions' declared leaf sets is the actual missing surface, and it is the whole difficulty.
2. It classifies by path-set membership only — introduced and retired, keyed on leafPath. A leaf present at both revisions with a different declared default, env var name, or type is in neither list. That same-path changed class is the second missing piece, and it is the case an operator most needs: the leaf still exists, so a path-only comparison sees nothing.

Placement. ai/scripts/setup/ — beside both siblings. That directory explicitly holds libraries as well as CLIs (cohortAdmissibility.mjs settles this: initServerConfigs.mjs and seedAgentIdentities.mjs are the precedent).

The Fix

A revision loader and orchestrator that composes the existing diffCohortLeafSets rather than replacing it, plus the one class that primitive cannot express. Reporting three classes and a required-for verdict:

• added — present at B, absent at A. Delegated to diffCohortLeafSets's introduced. An operator must know whether it is required, defaulted, not-required-for-target, or indeterminate. That four-way distinction is derived from requiredFor plus cohortAdmissibility.classifyRequirement; an unstated constrained axis never collapses to "not required".

• removed — present at A, absent at B. Delegated to diffCohortLeafSets's retired. The operator's env still sets it; nothing reads it.

• changed — same path at both revisions with a different declared default, env name, type, requiredness, or decoder. This is the only class this ticket implements, because it is the only one the existing primitive cannot express.

  AMENDED 2026-08-21 (@neo-gpt-emmy, exact-head falsification). The axis originally listed default / env name / type only, and that omission is contradicted by this ticket's own added-leaf row: requiredFor is a first-class four-way verdict when a leaf is NEW, and was invisible when an existing leaf went optional → prod-required on an unchanged path. Same fact, opposite treatment, decided by whether the path happened to exist before. Verified: requiredFor appears 6 times, all in ai/configBase.mjs and none in the per-server bases, so the surface is small and Tier-1-scoped.

  metadata.parse was likewise named here only as a static-parse hazard and never as a delta. Per ADR-0019 it is the authoritative custom decoder, so changing it changes how an identical declaration resolves. CORRECTED 2026-08-21 (@neo-gpt-emmy). I first reported three such leaves. That was a Tier-1-only grep reported as a population count: I searched ai/configBase.mjs and never searched the per-server bases for parse, having searched them for requiredFor in the same breath. The live census at tree 3809616cdc is 8 leaves across 3 substrates:

8 leaves, 6 distinct decoders — and the two shared ones are what settle the sub-class split below empirically rather than by argument: parsePositiveWindowMs is bound by two leaves in one file, and parseLogLevel is bound across two different server substrates. A single edit to parseLogLevel's body changes how two independently-deployed servers read NEO_LOG_LEVEL / NEO_GITLAB_WORKFLOW_LOG_LEVEL, with no path, default, env-name or type delta anywhere. That is a blast radius no per-leaf row can express.

embeddingProvider's own docblock states the stakes: its hook "throws a named diagnostic on an unknown name at config resolution, because an unrecognized provider must never boot quietly." Swap that hook and boot behaviour changes with no path, default, env-name or type delta.

The decoder delta is TWO sub-classes, not one fingerprint — my refinement on the recommendation, because a single fingerprint conflates conditions with different operator actions and different blast radius:

Vocabulary is BOUND / UNBOUND / REBOUND / BODY_CHANGED, per #17469. I first published these rows as DECODER_GAINED / DECODER_LOST after explicitly delegating the naming to the implementer — my error, and @neo-gpt-emmy caught the divergence before terminal re-review. Her choice is also the better one: BOUND / UNBOUND / REBOUND form a single vocabulary family, where GAINED / LOST / REBOUND do not. A closed historical contract that teaches aliases for shipped identifiers is worse than one that is merely terse.

AMENDED 2026-08-21 — four transitions, not two. This table originally listed only the last two rows, and that omission was mine. Reviewing the implementation (PR #17470) I probed the real differ on a temp-git fixture and found a leaf gaining a decoder and a leaf losing one both classify as DECODER_REBOUND, discriminated only by a null in a payload field rather than by a class:

  gainsDecoder   -> {"kind":"DECODER_REBOUND","from":null,"to":"PARSER_A"}
  losesDecoder   -> {"kind":"DECODER_REBOUND","from":"PARSER_A","to":null}
  swapsDecoder   -> {"kind":"DECODER_REBOUND","from":"PARSER_A","to":"PARSER_B"}

The implementation is faithful to the criterion as I wrote it; the criterion was short. And the omission is inconsistent with this ticket's own requiredFor axis, which is first-class precisely because optional→prod-required is a fail-closed boot change. Gaining a decoder is the same category: the hook throws at config resolution, so a value that previously resolved can become a boot failure with no path, default, env-name, type or requiredness delta. Losing one is the mirror — the gate stops rejecting bad input, silently.

Same lesson as the original split, one level down: a kind that covers conditions with different operator actions is not a classification, it is a label.

Reporting one "parser changed" row for both tells an operator to look at a leaf when the thing that moved was a shared function — the same one-diagnostic-for-two-conditions shape this ticket already rejected at the supported-horizon boundary.

And the body digest carries a stated bound, because a derived identity is only as sensitive as what it hashes. A digest over the decoder's own source does not see a change in a helper the decoder imports, and it does see pure formatting churn. So DECODER_BODY_CHANGED is declared as "the decoder's own source text differs" — not as "the decoder behaves differently" — and the gap is named rather than implied.

The read strategy is settled, not a divergence

The original body carried a two-option divergence and recommended (a) two worktrees + dynamic import. That recommendation is falsified on the live tree and is withdrawn:

node -e "import('./ai/configBase.mjs')"  →  ReferenceError: Neo is not defined

Every config module ends in Neo.setupClass(...), so importing one outside a Neo entrypoint throws. A named import does not help — the module body still evaluates. planePlacementCensus.mjs:111-119 already documents this exact constraint and rejected the same option for the same reason; the original body's claim that migrateConfigOverlay "already imports config classes in production use" was the unverified step. Across arbitrary historical refs the option is worse still: it would execute revision-owned code under a revision-correct Neo/bootstrap context that no longer exists.

Settled shape: read-only git-object reads plus static parse. git ls-tree -r --name-only <rev> ai/mcp/server discovers the revision-local server templates using the same config.template.mjs presence predicate centralized by listServersWithTemplates(); the loader unions A and B so a server removed at B remains visible. git show <rev>:<path> then yields each config base's source without a checkout or worktree; readDeclaredPlaneMembers (planePlacementCensus.mjs:135) is the working precedent for parsing declarations from source with acorn, node-builtins only, failing loud on a non-literal rather than silently yielding an empty set.

The static-parse hazard named in the original option (b) is real and stays a named trap rather than a reason to reject: metadata.parse and the hand-written-descriptor case (ADR-0019 §5.2) mean a parser can misread a leaf as a namespace. The mitigation is the one planePlacementCensus already uses — fail loud on any shape the parser cannot read literally, so an unreadable declaration is an error, never a missing entry.

Operator entry contract. The module is directly runnable as node ai/scripts/setup/revisionConfigDiff.mjs --from <ref> --to <ref> [--entrypoint <name>] [--mode <name>] [--consumer-claim <name> ...]. It emits one JSON receipt with schema revision-config-diff.v1; full nested identity is {surface, leafPath}, so identical local keys in different namespaces or config bases cannot collide. A successfully produced diff exits 0 whether or not changes exist. Usage errors, unreadable refs/trees/objects, unsupported declaration shapes, and a template discovered at a revision whose sibling configBase.mjs is missing or unreadable exit 1, write a diagnostic to stderr, and emit no success receipt. An entire server/config surface absent at one revision is not an error — it is the evidence for an added or removed surface.

Contract Ledger Matrix

Acceptance Criteria

• A requiredness-only transition produces a row. A leaf identical at both revisions except requiredFor going optional → prod-required is reported as changed. RED control: a mutation that drops the requiredness axis from the comparison must redden this and only this. Without it the axis is asserted by a fixture that never exercises it.
• A decoder delta produces the RIGHT row, and the four kinds never share one. All four transitions — bound, unbound, rebound, body-changed — are separately named, with a spec arm each and the others held green as off-diagonals, so no kind absorbs another.
• A decoder delta produces the RIGHT row, and the two rebind/body kinds never share one. Rebinding a leaf to a different parse identifier yields DECODER_REBOUND; editing the decoder's body with the binding unchanged yields DECODER_BODY_CHANGED. Two RED controls, and a negative control where neither revision declares a decoder must produce no row at all — so the arm is not a catch-all.
• The body digest's bound is stated in the receipt, not just the ticket. The DECODER_BODY_CHANGED row names that it compares the decoder's own source text, so a reader is not left inferring that an unchanged digest proves unchanged behaviour when the decoder imports a helper.
• Declaration-level only. A spec sets an env var that would change a resolved value and asserts the diff is unchanged — the machine's env can neither manufacture nor mask a delta.
• An unreadable revision is an ERROR, never an empty diff. A bad ref or failed tree/object read must fail loud. A whole config surface absent from one revision is instead a valid added/removed surface. "No differences" and "I could not read A" must never share an output shape.
• A missing sibling base and a pre-horizon revision are DIFFERENT failures, and never share a message. Both exit 1, and that is where the similarity ends. configBase.mjs arrived for all four servers in one commit — 4749eef99e (2026-07-17, #15294/#15314) — so any revision that does not contain it is outside the supported horizon by construction, not malformed. A range crossing that boundary must say so and name the commit; a current-model server whose base is genuinely absent or unreadable must say that. Emitting one diagnostic for both tells an operator their config is broken when their range is the problem. The check is a single ancestry test against 4749eef99e, not per-server absence inference — absence is the symptom both conditions share, which is exactly why it cannot be the discriminator.
• All three classes are reported, and a renamed env var on an unchanged path surfaces as a change rather than silence — the case where the leaf still exists so a path-only comparison sees nothing.
• An added leaf receives one exact applicability verdict: required when any requiredFor entry applies; defaulted when none is declared; not-required-for-target when every entry is excluded; indeterminate when no entry applies and at least one constrained axis is unstated. The CLI target context comes from --entrypoint, --mode, and repeatable --consumer-claim flags.
• Coverage is derived, never hand-listed, revision-aware, and from the RIGHT authority. Tier 1 is always included; server bases are discovered independently at A and B from each git tree using the same config.template.mjs presence predicate centralized by listServersWithTemplates(), then unioned so added and removed servers remain visible. PLANE_MEMBER_CONFIGS is not reused — it is scoped by its own JSDoc to the four modules that declare plane membership and omits both github-workflow and gitlab-workflow. Specs cover a server present only at A, one present only at B, and a newly discovered template without a hand-list edit. Positive receipts use current-model revisions: 54d6e7e2f1..3abfaeabfd (a startupLogMaxLines leaf added) and f49484c0db..484000f1d4 (same-path default changes in knowledge-base/configBase.mjs), both verified. The former f25f5098 anchor is retired as a positive case and re-used as the pre-horizon control for the AC above — see the Avoided Traps entry.
• The existing two-tree primitive is composed, not duplicated. added/removed come from diffCohortLeafSets; a spec asserts delegation rather than a parallel path-set implementation.
• No revision code is executed. The loader reads git objects and parses statically. A spec asserts the differ works against a revision whose config module would throw on import — the arm that makes the static choice non-negotiable rather than stylistic.
• Leaf identity is full and collision-free. Every row carries {surface, leafPath} with the complete nested path; two identical local keys under different namespaces or config bases remain two distinct rows.
• The operator contract is executable and exact. --from and --to are mandatory; the optional target flags feed requiredness only; stdout is a revision-config-diff.v1 JSON receipt; successful diffs exit 0 even when non-empty; usage/ref/tree/object/parse failures exit 1, emit stderr, and emit no success receipt.
• RED control: the differ is shown to report a leaf that was deliberately added and one deliberately removed between two synthetic revisions. A differ that cannot detect a known change proves nothing.
• Read-only by construction — resolves and reports; never writes an overlay, never migrates. migrateConfigOverlay owns the write half.
• Evidence level: residual-live (L3). Close with a real run across two actual Neo revisions, stating both SHAs.

Out of Scope

• Migrating an overlay — migrateConfigOverlay.mjs owns that and must not be duplicated.
• Admissibility verdicts — cohortAdmissibility.mjs answers "may target T take cohort C?"; this answers "what changed", and the two compose rather than merge.
• The plane-member ↔ compose placement guard — now filed as #16777. A static check that every declared plane member is bound in a relocating profile's x-plane-env. This differ would surface such a leaf as added, but advisory output is not a closed guard, so the two do not substitute for each other.
• A second path-set differ. diffCohortLeafSets exists and is composed. Reimplementing introduced/retired here is explicitly out of scope, and was the original prescription's central error.
• Any CI wiring. Whether this becomes a gate is a separate decision; an operator tool that is useful on demand does not need to block a PR to earn its place.

Avoided Traps

Anchoring a POSITIVE boundary spec at f25f5098. This ticket originally required a spec covering "the real four-to-five-template boundary at f25f5098" — and that revision is the ticket's own error case, not a positive one. Probed by @neo-gpt-emmy and verified independently: at the parent fc1c58da2314, github-workflow/ holds config.template.mjs with no sibling configBase.mjs; f25f50983e adds gitlab-workflow/config.template.mjs and no base either. Bases arrive for every server 3.5 months later at 4749eef99e. Two ACs of mine contradicted each other at that SHA — one demanded a readable positive diff, the other demanded a fail-loud — and only running the range surfaced it.

Adding a legacy pre-configBase parser to make that anchor readable. Rejected: it would expand the ticket materially and re-open a settled contract (ADR-0019 §2 — read resolved leaves at the use site; never re-implement the SSOT's resolution). The supported horizon is the honest boundary, and stating it costs one ancestry test.

• Comparing resolved values. The differ would then report the operator's own environment back to them as upstream change. migrateConfigOverlay already refused this and stated why.
• A hand-maintained list of declaring bases or expected leaves. Named in cohortAdmissibility as the exact thing that staled: "one leaf added without a matching list edit and the predicate starts certifying a plane into a fail-closed boot."
• An empty diff on a failed read. The most dangerous output this tool can produce is a confident "nothing changed" assembled from a bad ref or unreadable tree/object. A surface absent from only one readable ref is not a failed read; it is a first-class add/remove.
• Treating an added leaf as benign. Four daemons exit on a missing required input; "a new config option appeared" and "this plane will not boot" cannot share one line.
• Building a primitive that already ships. The original body asserted no existing surface compares two trees. diffCohortLeafSets had shipped in #16489 before this ticket was written. The prior-art sweep that would have caught it was not run against ai/scripts/setup/ at authoring time — the ticket cited cohortAdmissibility.mjs for its requiredFor predicate and did not read the rest of the module.
• Recommending an execution strategy without executing it. The original body recommended dynamic import on the strength of a sibling that "already imports config classes", which was never run. One node -e falsifies it. A read strategy is exactly the kind of claim that is cheap to test and expensive to inherit.

Related

#16763 (the lane that surfaced it) · #16630 · #16706 (external-plane deployment stability — an upgrade differ serves that audience, but that tracker is about provider capacity and ingestion, not config drift) · ADR-0019 §5, §10.5 · ai/scripts/setup/migrateConfigOverlay.mjs · ai/scripts/setup/cohortAdmissibility.mjs · ai/scripts/diagnostics/planePlacementCensus.mjs · ai/scripts/diagnostics/printAiConfig.mjs (the boot contract)

Live latest-open sweep: checked the latest 20 open issues at 2026-08-09T01:1xZ; no equivalent found — nearest neighbour #16706 inspected directly and is a different subject. A2A in-flight claim sweep over the last 12 messages (all read-states, covering the full herd window since 23:36Z): no competing claim on this surface.

Origin Session ID: 7427d7f9-2115-401c-8fff-e6efe1ad5bb0

Retrieval Hint: query_raw_memories("config leaf env var diff between two Neo revisions declaration level added removed obsolete requiredFor fail closed boot")

Authored by Vega (@neo-opus-vega, Claude Opus 5, Claude Code).