LearnNewsExamplesServices
Frontmatter
id16587
titleIn-process ingest params are absent from the contract that gates them
stateClosed
labels
bugaiarchitecture
assignees[]
createdAtAug 6, 2026, 12:03 PM
updatedAtAug 6, 2026, 1:03 PM
githubUrlhttps://github.com/neomjs/neo/issues/16587
authorneo-opus-vega
commentsCount0
parentIssue16566
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtAug 6, 2026, 1:03 PM

In-process ingest params are absent from the contract that gates them

Closed Backlog/active-chunk-13 bugaiarchitecture
neo-opus-vega
neo-opus-vega commented on Aug 6, 2026, 12:03 PM

Sub-ticket of epic #16566.

Context

Pull-mode tenant ingestion embedded correctly and then rejected itself. #16580 (PR #16578) added the diagnostic that inverted the diagnosis: create-app reported ingested=50 embeddings=50 errors=0 and then failed with KB_TENANT_REPO_SYNC_EMPTY_MATERIALIZATION. The effect was real; the proof was missing.

The cause is not in either service. ai/services.mjs:139 parses every Knowledge Base service call through a Zod schema built from ai/mcp/server/knowledge-base/openapi.yaml:

const parsedArgs = zodSchema.parse(args || {});   // Zod DROPS undeclared keys, silently

ingestSourceFiles documents materializationAttempt (IngestionService.mjs:181) and viaMcp (:182-186), and its in-process callers pass both. Neither is declared in the contract. Both are deleted from the payload before the method runs — no error, no log, no summary entry.

Measured in the running orchestrator with the real create-app envelope:

attempt sent in payload : {"attemptId":"a7a07f76601f638c0af6286dfcaaa85d","ingestContractVersion":2}
producer received       : materializationAttempt: undefined
                          manifestSnapshot: present paths=50
                          summary.ingested: 50   summary.errors: 0

manifestSnapshot is declared, so it survives. That asymmetry is the whole signature: the manifest persists with 50 correct paths while the receipt never appears.

The Problem

One mechanism, both configured repos, two different symptoms:

repo stripped param consequence
create-app materializationAttempt attempt is null → the receipt block never runs → a successful materialization persists no proof → assertFullMaterializationEffect raises EMPTY_MATERIALIZATION against its own ingest
neo viaMcp payload.viaMcp !== false re-reads as trueVectorService.mjs:1098 work-volume gate throws → KB_VECTOR_EMBED_FAILED, embeddings=0 on 24,590 chunks

create-app sits at exactly the 50-chunk mcpSyncMaxChunks default and slips under the gate; neo does not. That is why the two failed differently from a single cause.

Corroborating sweep: no materializationReceipt has ever existed in this deployment — zero across every graph node and the full GraphLog. The receipt feature has never once produced its proof since it shipped in #16047.

The Architectural Reality

buildZodSchema is shared by two consumers of one contract:

  • ai/services.mjs:132 — the in-process validation gate. An undeclared param is silently dropped.
  • ai/mcp/ToolService.mjs:167 — the agent-facing tool input schema. An undeclared param is invisible to agents.

Keeping those two shapes identical is the reason in-process calls route through the validated facade at all, since agents can invoke the same operations manually. The Zod layer is correct; the contract was incomplete.

The agent-facing boundary is already defended independently of the schema — ingestSourceFilesTool.mjs:94 spreads viaMcp: true after caller args, :96 deletes materializationAttempt inbound, :106 deletes materializationReceipt outbound. Enumerated callers of ingestSourceFiles across ai/: TenantRepoSyncService:1309 (pull, in-process), ingestSourceFilesTool:98 (MCP, protected), ingestTenant.mjs:210 (CLI). MCP is the only agent-reachable path and it is the protected one.

The Fix

Declare both params on IngestSourceFilesRequest in ai/mcp/server/knowledge-base/openapi.yaml. No code change.

Rejected: making the Zod schema passthrough. That would fix the symptom by destroying the property the facade exists to hold — the in-process shape and the agent-facing tool shape would be free to diverge silently.

Contract Ledger Matrix

Target Surface Source of Authority Proposed Behavior Fallback Docs Evidence
IngestSourceFilesRequest.materializationAttempt IngestionService.mjs:181 JSDoc; TenantRepoSyncService:1278-1283 mints it Declared; survives the gate none needed schema description producer received undefined, measured live
IngestSourceFilesRequest.viaMcp IngestionService.mjs:182-186; VectorService.mjs:1098 gate Declared; explicit false preserved MCP dispatch forces true schema description neo at 24,590 chunks → KB_VECTOR_EMBED_FAILED
agent-facing tool shape ToolService.mjs:167 Unchanged — tool deletes/overrides regardless ingestSourceFilesTool.mjs:94,96,106

Decision Record impact

none. Two schema properties declared to match existing documented service parameters. No ADR authority is amended or challenged.

Acceptance Criteria

  • materializationAttempt and viaMcp are declared on IngestSourceFilesRequest.
  • Both survive buildZodSchema(doc, ingestOp).parse(...); an explicit viaMcp: false is preserved rather than defaulted away.
  • Regression spec is RED with the declarations removed, and confirmed executing by name rather than inferred from a suite count.
  • The spec parses through the real buildZodSchema rather than a hand-built schema, so it exercises the deployed gate.
  • Every spec that reads this contract still passes, including advertisedSurfaceDigest (a tool's input schema is part of the digest).
  • The agent-facing surface is unchanged: an agent cannot supply materializationAttempt, cannot observe materializationReceipt, and cannot set viaMcp: false.
  • (post-merge) A live create-app sweep mints a receipt and commits its checkpoint; the following sweep is a no-op diff rather than a full re-ingest.
  • (post-merge) neo embeds rather than raising KB_VECTOR_EMBED_FAILED.

Out of Scope

  • The zero-chunk disposition. A repo that genuinely materializes zero chunks still raises EMPTY_MATERIALIZATION and still backs off permanently after this change — provesUncommittedRetry requires an attempt id differing from the current one, so with hasEffect false the throw stands. That is #16577 ACs 1/2/5 and stays open there.
  • The misleading EMPTY_MATERIALIZATION message text — #16577 AC 4.
  • A mechanical guard for this defect class — #16585.
  • The destructive delete-upfront default — #16584.

Related

  • Epic #16566 · #16577 (the sibling defect whose motivating observation this work falsified) · #16580 / PR #16578 (the diagnostic that inverted the diagnosis) · #16581 / PR #16579
  • #16047 — where materializationAttempt shipped undeclared
  • #16584 · #16585 — filed from the same investigation

Origin Session ID: 6004a4aa-2089-4b14-b73f-b58c08cf53d9

Retrieval Hint: query_raw_memories("Zod strips undeclared service params ingest_source_files contract") · ai/services.mjs:139 · VectorService.mjs:1098

Authored by @neo-opus-vega (Claude Opus 5).

tobiu referenced in commit c17b562 - "fix(kb): an undeclared service param is stripped before the method sees it (#16587) (#16583) on Aug 6, 2026, 1:03 PM
tobiu closed this issue on Aug 6, 2026, 1:03 PM