LearnNewsExamplesServices
Frontmatter
id16115
titleRe-summarization preserves graphDigested and skips new session turns
stateClosed
labels
bugairegressionarchitecture
assigneesneo-gpt
createdAt7:55 AM
updatedAt1:44 PM
githubUrlhttps://github.com/neomjs/neo/issues/16115
authorneo-gpt
commentsCount0
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAt1:44 PM

Re-summarization preserves graphDigested and skips new session turns

Closed Backlog/active-chunk-10 bugairegressionarchitecture
neo-gpt
neo-gpt commented on 7:55 AM

Context

The live post-merge investigation for #16017 exposed a second defect while falsifying receipt replay in #16114.

Session summaries and Dream lifecycle state share one Chroma row. SessionService.summarizeSession() upserts a new document and synthesis metadata under the stable summary_<sessionId> id. DreamService later extends that row with graphDigested:true and digestState:'digested'.

A disposable production-Chroma witness now proves that a subsequent upsert of the synthesis metadata preserves those unrelated Dream-owned fields. The current Knowledge Base answer assumed whole-metadata replacement; the real adapter behavior falsifies that assumption.

This matters beyond receipt verification. A session with new raw turns is deliberately re-summarized, but the surviving graphDigested:true keeps it out of DreamService.findUndigestedSessions(). The new turns never enter the next deterministic memory/session ingestion and semantic extraction pass.

The Problem

graphDigested is an eternal boolean attached to a mutable session.

The writer and consumer lifecycle is:

  1. SessionService.findSessionsToSummarize() detects raw-memory drift.
  2. SessionService.summarizeSession() writes a new summary/base metadata to the existing Chroma id.
  3. Chroma preserves the previous Dream overlays.
  4. DreamService.addUndigestedRowsFromBatch() rejects every row whose preserved graphDigested is true.
  5. The newly added turns remain outside the Dream pass even though the flag only attests to an older input frontier.

Source inspection also corrects the easy framing error: Dream does not digest the summary prose. Before extraction, DreamService replaces session.document with the current raw episodic turn documents. The stale flag therefore suppresses ingestion of new raw turns; this is not merely a stale digest of changed LLM prose.

A bare reset during re-synthesis would repair the common ordering but not the concurrency contract. Dream can begin processing revision A, synthesis can publish revision B, and Dream can then write graphDigested:true for A onto the shared row. The boolean has no way to express which input it certifies.

The Architectural Reality

  • SessionService.mjs owns summary synthesis and already has the exact raw-memory input set used to detect drift.
  • DreamService.mjs owns REM eligibility and the completion marker after memory/session ingestion, tri-vector extraction, topology, and gap inference.
  • MemorySessionIngestor.syncSessionToGraph() consumes the current raw turn set. Its computeSessionPayloadHash() covers only a small metadata projection and does not fence the Dream run against a concurrent turn-frontier change.
  • sessionSummaryReceiptStore.mjs owns synthesis receipts. After #16114, any new synthesis-owned metadata field must be declared at issuance and verified by replay.
  • A completion receipt may only certify the input revision actually processed. Mutable work requires revision equality, not an unqualified historical boolean.

The Fix

Replace the eternal boolean gate with an input-revision contract:

  1. Session synthesis publishes a deterministic revision for the raw episodic input frontier it observed. The revision must change when the Dream-relevant turn set/content changes and remain stable for an equivalent input.
  2. Dream captures that revision with the row it selects and records the revision it actually completed.
  3. Eligibility is revision equality: a row is current only when its completed Dream revision equals its current synthesis/input revision. Missing or mismatched completion is pending.
  4. A stale Dream completion for revision A cannot mask a later revision B; writing A after B still leaves the row eligible.
  5. Preserve a bounded compatibility path for existing graphDigested rows, with an explicit retirement condition. Do not create an unbounded second lifecycle.
  6. Add deterministic interleaving coverage plus a disposable-Chroma witness for metadata merge behavior.

The exact revision representation is an implementation decision after intake. It must fence the raw turn input Dream consumes; memoryCount alone is not sufficient.

Contract Ledger Matrix

Target Surface Source of Authority Proposed Behavior Fallback Docs Evidence
Session input revision SessionService drift/synthesis path Publish a deterministic Dream-input revision with the synthesis row Fail loud rather than mint an ambiguous revision Method JSDoc Same-input stability and changed-input tests
Dream eligibility DreamService.findUndigestedSessions() Pending when completed revision is absent or differs from current input revision Legacy compatibility is bounded and observable REM lifecycle JSDoc Selection matrix
Dream completion DreamService.processUndigestedSessions() Attest only the captured input revision that completed every required phase Failure leaves that revision unacknowledged Completion JSDoc A/B interleaving test
Synthesis receipt sessionSummaryReceiptStore Carry and strictly verify the synthesis-owned current revision Existing replay failure semantics Receipt key-set JSDoc Receipt round-trip/replay test
Chroma merge behavior Production adapter Foreign overlays may survive base upserts without hiding revision mismatch No delete/recreate workaround Test fixture comment Disposable-Chroma witness

Decision Record impact

none expected — this replaces an ambiguous lifecycle boolean with a fenced revision inside the existing SessionService → DreamService contract. If intake finds that the revision must become a broader cross-projection protocol, graduate that design question before implementation.

Acceptance Criteria

  • A re-summarized session whose Dream-relevant raw input changed is eligible even when legacy graphDigested:true / digestState:'digested' metadata survives the Chroma upsert.
  • Dream completion identifies the exact input revision processed; it is not an unqualified boolean.
  • Interleaving proof: Dream starts revision A, synthesis publishes revision B, Dream completes A; B remains eligible.
  • Equivalent raw input produces a stable revision and does not create needless re-digestion.
  • The revision covers Dream-relevant turn identity/content; a count-only collision is rejected.
  • The newly declared synthesis field passes #16114 receipt issuance, replay, and ownership checks.
  • Existing rows have a bounded compatibility/migration behavior plus a named retirement trigger for the legacy boolean gate.
  • Deterministic unit coverage and a disposable-Chroma integration witness reproduce metadata merge semantics.
  • Post-merge validation proves one naturally re-summarized session with a surviving old overlay re-enters Dream and reaches current-revision completion.

Out of Scope

  • Deleting and recreating healthy Chroma rows to force metadata replacement.
  • Making the synthesis receipt own mutable Dream fields.
  • Re-running Dream because only nondeterministic summary prose changed while the raw input frontier remained equivalent.
  • Solving historical semantic-edge pruning beyond the revision/eligibility contract; if re-processing exposes a separate superseded-edge reconciliation defect, file it with its own graph falsifier.
  • Closing #16017 before receipt recovery and this lifecycle follow-up have separate runtime evidence.

Avoided Traps

  • Reset graphDigested:false and stop. This leaves the A/B late-completion race.
  • Use memoryCount as the revision. Equal counts do not prove equal turn identity or content.
  • Hash only summary prose. Dream consumes raw episodic turns, not the summary document.
  • Delete/re-add the Chroma row. Storage replacement semantics are not a lifecycle protocol.
  • Let Dream absorb synthesis ownership. The producer publishes input revision; the consumer attests completion of that revision.
  • Keep both boolean and revision forever. Compatibility must carry a retirement condition.

Duplicate / Claim Sweep

Knowledge Base, Memory Core, and committed-source sweeps found #12065, #13624, #13647, and #13697 as adjacent REM/backlog/projection history. None defines input-revision invalidation for re-summarized sessions. The KB's replacement-semantics premise was directly falsified by disposable Chroma.

Live GitHub all-state searches for graphDigested resummarize invalidation and Dream digest stale session summary found no equivalent. The latest 20 open issues were checked created-descending at 2026-07-29T05:53Z; no competing ticket exists.

A2A all-state concept sweep found only the #16114 review thread that surfaced this follow-up; no competing claim or ticket intent exists.

Related

Related: #16017 Related: #16114 Related: #16105 Related PR: #16110 Related: #12065

Origin Session ID: 019fac51-ddcb-7212-902e-09d3a9d19098

Handoff Retrieval Hints

  • query_raw_memories("graphDigested re-synthesis Dream input revision stale raw turns")
  • ai/services/memory-core/SessionService.mjs
  • ai/daemons/orchestrator/services/DreamService.mjs
  • ai/services/ingestion/MemorySessionIngestor.mjs
  • Disposable-Chroma witness: sessionSummaryReceiptStore.spec.mjs

Authored by Euclid (GPT-5 Codex, Codex Desktop).

tobiu referenced in commit 20913af - "fix(ai): fence Dream input revisions (#16115) (#16127) on 1:44 PM
tobiu closed this issue on 1:44 PM