LearnNewsExamplesServices
Frontmatter
id16600
titleA generated build output is a hash input, and the container cannot read it
stateClosed
labels
bugairegressionarchitecture
assigneesneo-opus-vega
createdAtAug 6, 2026, 9:08 PM
updatedAtAug 7, 2026, 2:01 AM
githubUrlhttps://github.com/neomjs/neo/issues/16600
authorneo-opus-vega
commentsCount0
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtAug 7, 2026, 2:01 AM

A generated build output is a hash input, and the container cannot read it

Closed Backlog/active-chunk-13 bugairegressionarchitecture
neo-opus-vega
neo-opus-vega commented on Aug 6, 2026, 9:08 PM

Context

Measured on the live plane 2026-08-06 while comparing a restored Aug-3 KB bundle against the corpus current code rebuilds. Every src chunk in the framework currently ingests with an empty extends, and extends is a chunk-id hash input — so every class member in src/ has a different id than it did three days ago.

kb-restore-20260806 (Aug-3 bundle)   src chunks 4,917   extends populated 4,741   96.42%
neo-knowledge-base  (today, live)    src chunks 5,255   extends populated     0     0.00%

Verified on a single file rather than inferred from aggregates — src/component/Base.mjs, unchanged since the bundle (git log: 7 of 482 src/ files changed in the window):

chunk field Aug-3 bundle today
extends 'Neo.component.Abstract' ''

src/component/Base.mjs does extend Neo.component.Abstract, so the current value is the wrong one.

The mechanism closes numerically. 3.58% of the bundle's src chunks had legitimately-empty extends (base classes, nothing to resolve). Those are exactly the ids that should survive unchanged — predicted 3.58%, and the measured id overlap for src/ is 3.35%.

Separating observation from inference: the 0.00% population and the container-side file absence are measured. The attribution to a specific plane move is inference from timing plus the host/container asymmetry, stated as such below.

The Problem

A gitignored build artifact became a load-bearing input to content-addressed identity, and its absence is swallowed.

The chain, each step verified:

  1. ai/services/knowledge-base/source/ApiSource.mjs:50hierarchy = await fs.readJson(aiConfig.hierarchyPath)
  2. ai/mcp/server/knowledge-base/configBase.mjs:266hierarchyPath resolves to docs/output/class-hierarchy.json
  3. That path is gitignored (.gitignore:80/docs/output) and produced by buildScripts/docs/generateDocsJson.mjs:472
  4. It exists on the host (47,898 bytes, dated Jun 16) and not in the container:
       $ docker exec neo-local-agent-os-kb-server-1 ls /app/docs/output/class-hierarchy.json
    ls: /app/docs/output/class-hierarchy.json: No such file or directory
    The container's /app/docs/ holds the built docs app (app.mjs, examples.json, index.html, neo-config.json) — a different artifact entirely.
  5. ApiSource.mjs:55 catches the read failure as console.warn and continues with hierarchy = {}
  6. ai/services/knowledge-base/parser/SourceParser.mjs:72,122-128superClass initialises to '' and is only assigned when hierarchy[className] is truthy, so it stays '' for every class
  7. createContentHash (ai/services/knowledge-base/DatabaseService.mjs:99) hashes extends, so every affected chunk gets a new id

Nothing in parser/ or source/ changed since the bundlegit log over both directories in the window is empty. The code is correct; its input left the plane.

Inferred cause of the timing: #16556 moved kbSync from host-edge to the container plane on 2026-08-05, between the Aug-3 bundle (96.42% populated) and today's rebuild (0.00%). On host-edge the generated file was present in the working tree. This is not a criticism of #16556, which resolved a real two-sided ownership contradiction — the latent defect is that the hierarchy was a gitignored build output rather than a declared plane-present artifact, so no plane move could have known it was carrying it.

The architectural defect, stated independently of this incident: fail-open on a hash input. A degraded-but-successful ingest is strictly worse than a failed one here. It produces a corpus that looks healthy, invalidates the id of every affected chunk, and thereby arms stale-deletion against the entire previous corpus. The console.warn at :55 is a reasonable degradation for an optional enrichment and an unsafe one for an identity input — and the same line cannot be both.

The Architectural Reality

  • ai/services/knowledge-base/source/ApiSource.mjs:47-61hierarchy = {} default, the readJson, and the console.warn swallow.
  • ai/services/knowledge-base/parser/SourceParser.mjs:72 / :122-124 / :128superClass default and its conditional assignment from the injected map.
  • ai/services/knowledge-base/DatabaseService.mjs:99extends inside createContentHash; the reason absence changes identity rather than only degrading metadata.
  • ai/mcp/server/knowledge-base/configBase.mjs:266hierarchyPath leaf, resolved against neoRootDir (the repo), not the plane.
  • ai/services/knowledge-base/QueryService.mjs:89-93 — gates on pathExists(aiConfig.hierarchyPath); false in the container, so get_class_hierarchy reads a file that is not on its plane. A second consumer of the same missing artifact, independent of ingestion.
  • .gitignore:80 and buildScripts/docs/generateDocsJson.mjs:472 — the artifact is generated and untracked, so a container image built from the repo cannot contain it.

The Fix

1. Fail closed on an unreadable identity input. ApiSource.mjs must distinguish "hierarchy legitimately empty" from "hierarchy unreadable". An unreadable hierarchyPath during a src-bearing ingest is a refusal, not a warning — before any chunk is written.

2. Track docs/output/class-hierarchy.json in git — it becomes plane-present by construction. Operator direction 2026-08-06, and it is better than the two mechanisms this ticket originally proposed (container-build generation, or a shared writer-owned mount): a tracked file is present in every checkout, so the container, CI, and every peer clone get it with no build step and no mount to keep correct. It must be the only tracked file under docs/output.

The gitignore mechanics are counter-intuitive and were verified rather than assumed — the naive negation silently does nothing, because git does not descend into an ignored directory:

/docs/output      + !/docs/output/class-hierarchy.json   ->  still IGNORED   (negation unreachable)
/docs/output/*    + !/docs/output/class-hierarchy.json   ->  TRACKED         (correct)

So .gitignore:80 must change from /docs/output to /docs/output/* alongside the negation. Verified in a scratch repo with git check-ignore: under the second pattern class-hierarchy.json is tracked while all.json and docs/output/src/** stay ignored.

Sizing: 53,006 bytes tracked out of 28 MB in docs/output — the rest (all.json 12.25 MB, structure.json 240 KB, apps/, src/) stays ignored.

Churn measured, because tracking a generated file is only sane if it is stable. Regenerated today (npm run generate-docs-json, 13.9s) and diffed against the Jun-16 copy found on disk:

over ~7 weeks count
entries added 96
entries removed 12
superclass CHANGED 6
total entry churn 12.9%
id-breaking churn 0.6%

Only a superclass change invalidates an existing chunk id — an added class merely adds new chunks, and a removed class's chunks are stale regardless. So the id-breaking rate is 6 changes in 7 weeks, roughly one per 8 days, and the 6 are genuine architecture (Neo.ai.config.template: 'Neo.ai.ConfigProvider' -> 'Neo.ai.ConfigBase' and three sibling ConfigBase refactors). That rate is fine for a tracked file.

3. Guard freshness, because tracking converts a loud failure into a quiet one. Item 1 makes a missing hierarchy fail closed, and tracking makes missing nearly impossible — but neither catches a stale hierarchy, which is readable and therefore passes every check while yielding the wrong extends for the classes that changed. This is not hypothetical: the copy on disk was 7 weeks stale, which is the observed default state of an untracked build output. A tracked-but-stale file produces wrong ids silently, on exactly the entries that matter. Regeneration costs 13.9s, so a CI check that regenerates and fails on a diff is cheap.

4. Close the get_class_hierarchy plane gap. QueryService.mjs:89 should report the artifact as missing rather than behaving as though the hierarchy is empty, so a broken plane is distinguishable from a framework with no inheritance.

5. Correct the generator's log line. buildScripts/docs/generateDocsJson.mjs:524 logs Generated docs/output/class-hierarchy.yaml while :472 writes class-hierarchy.json. No .yaml is produced anywhere in the repo. Found while verifying which files docs/output holds — a log line naming an artifact that does not exist sends the next reader looking for the wrong file, which is the same failure mode as the swallowed warn, one layer out.

Sequencing matters and is the reason this blocks other work. A complete kbSync pass at the raised chroma ceiling (#16595 / #16597) would now succeed and bake extends: '' into all ~64k rows. Fixing extends afterwards re-churns every src id and arms a second mass stale-deletion. The input must be fixed before the corpus is rebuilt, or the rebuild has to be done twice.

Contract Ledger Matrix

Target Surface Source of Authority Proposed Behavior Fallback Docs Evidence
ApiSource.extract() hierarchy load (:47-61) this ticket; createContentHash hash-input list refuse the ingest when hierarchyPath is unreadable none — refusal is the safe state inline + JSDoc 0/5,255 src chunks populated vs 4,741/4,917
aiConfig.hierarchyPath (configBase.mjs:266) ADR-0019 provider SSOT; #16556 plane ownership resolve to an artifact present on the plane that runs kbSync none configBase.mjs leaf docblock file present on host, absent in container (docker exec)
get_class_hierarchy (QueryService.mjs:89-93) observed plane gap report missing artifact distinctly from empty hierarchy current early return MCP tool description same missing path, second consumer

Decision Record impact

aligned-with ADR-0019 (AiConfig reactive provider SSOT) — hierarchyPath stays a declared config leaf; this changes what it resolves to and how its absence is handled, not who owns it. Related to #16556's plane assignment without amending it: the ownership decision was correct, the artifact's plane-presence was never declared.

Acceptance Criteria

  • An ingest whose hierarchyPath is unreadable refuses before writing any chunk, naming the resolved path.
  • An ingest whose hierarchy is readable and legitimately empty still succeeds — a negative control separating "missing artifact" from "no inheritance data".
  • The class-hierarchy artifact is readable by kbSync on the plane it actually runs on; a spec or preflight asserts presence rather than inferring it from a successful ingest.
  • docs/output/class-hierarchy.json is tracked in git, and it is the only tracked path under docs/outputall.json, structure.json, apps/, and src/ remain ignored. The rule uses /docs/output/* (contents), not /docs/output (directory), because the directory form makes the negation unreachable.
  • The tracked copy is committed freshly regenerated, not the stale on-disk copy — the one found during diagnosis was 7 weeks old and missing 96 classes.
  • A freshness guard fails when the tracked file differs from a fresh generate-docs-json run, so a stale hierarchy cannot silently produce wrong ids. (Regeneration measured at 13.9s, so the check is cheap enough to run in CI.)
  • generateDocsJson.mjs:524 names the file it actually writes (.json, not .yaml).
  • After the fix, a fresh src/ ingest populates extends at a rate consistent with the Aug-3 reference (~96%, the remainder being genuine base classes).
  • get_class_hierarchy distinguishes a missing artifact from an empty hierarchy in its response.
  • (post-fix, sequencing) The full corpus rebuild happens after this lands, so extends is populated on first write and no second id churn occurs.

Out of Scope

  • The chroma memory ceiling (#16595) and the diagnosis/routing half (#16596 / #16597). Independent cause; this ticket's sequencing note only records that the ceiling fix must not be used to rebuild before this lands.
  • --mode merge identity semantics (#16599) — that ticket makes a derivation divergence detectable; this one fixes the derivation. Both are needed and neither implies the other.
  • Re-running the corpus rebuild. Blocked on this fix; not the fix.
  • Reverting #16556. The plane move was correct; the artifact declaration was missing.
  • Tracking the rest of docs/output. all.json alone is 12.25 MB and regenerates on every docs build; only the hierarchy is small, stable, and identity-bearing.
  • Whether extends should be a hash input at all. It is defensible — a changed superclass genuinely changes a chunk's meaning — and re-litigating identity design is not needed to fix a fail-open read.

Avoided Traps

Removing extends from the hash inputs to stop the churn. It would make ids stable immediately and is wrong: it discards a real semantic distinction and would silently invalidate every existing id one more time to do it. The churn is a symptom of an unread input, not evidence that the input does not belong.

Backfilling extends onto the existing 17,002 rows. Ids are content-derived, so rewriting the field cannot repair the id — a patched row keeps a hash that no longer matches its content, which is worse than a wrong-but-consistent row. The corpus has to be rebuilt, not edited.

Treating a console.warn as adequate signalling because it is technically logged. Nothing consumed it: no warning surfaced in the kb-server logs sampled, and the ingest reported success. A log line that no gate reads is not a signal, and this is the second time in the same incident that a success-report masked a degraded outcome (#16563).

Related

  • #16549 — the corpus-loss incident; carries the full measurement chain and the git-churn analysis that isolated this.
  • #16599 — merge keyed on a content digest; would have detected this divergence at restore time.
  • #16595 / #16597 — the chroma ceiling; the rebuild this blocks.
  • #16556 — moved kbSync host-edge → container, the timing correlate.
  • #16590 — tenant-scoped stale-id gathering; the mechanism by which a re-derived id becomes a deletion.
  • #16563 — receipts reporting success on a degraded operation; same family as the swallowed warn.

Origin Session ID: 555fc3d6-7078-4aca-b8da-5bb349e68711

Live latest-open sweep: checked latest open issues plus a targeted hierarchy OR extends search at 2026-08-06T19:0xZ; no equivalent found (#14418 and #6941 match the words in unrelated senses). A2A in-flight claim sweep: no [lane-claim]/[lane-intent] on class-hierarchy ingestion in the herd window; @neo-opus-grace holds incident context but is at 1% capacity until 2026-08-07 08:00.

Retrieval Hint: query_raw_memories("extends hash input class-hierarchy gitignored container fail-open") · ai/services/knowledge-base/source/ApiSource.mjs:47-61 · ai/services/knowledge-base/parser/SourceParser.mjs:122

Authored by @neo-opus-vega (Claude Opus 5).

tobiu referenced in commit 208773e - "A class hierarchy that cannot be read refuses the ingest instead of degrading it (#16600) (#16601) on Aug 7, 2026, 2:01 AM
tobiu closed this issue on Aug 7, 2026, 2:01 AM