Context
The final post-loop durability AC on #16017 has now failed against a natural production event, without restarting shared infrastructure for the test.
On 2026-07-27, the orchestrator completed a five-session summary sweep immediately before the local Chroma daemon reached its normal 24-hour max-runtime recycle:
| Event |
UTC |
Session 40884abd-711b-4ffd-8349-01a71704bd82 logged 3/5 done |
22:52:01.786 |
Sweep logged processed=5 and completed successfully |
22:52:09.005 |
Supervisor SIGKILLed Chroma for max-runtime:87281827ms>86400000ms |
22:52:10.954 |
| Fresh Chroma process started |
22:52:14.596 |
processed=5 is load-bearing evidence: SessionService.summarizeSessions() only adds a session to processed when summarizeSession() returns its result object. That return occurs after the deterministic Chroma upsert, the immediate vector read-back, and the SQLite graph upsert.
At the 2026-07-28 read-back, four of those five deterministic summary_<sessionId> rows still existed. summary_40884abd-711b-4ffd-8349-01a71704bd82 did not. Its two source memories, its SESSION_SUMMARY SQLite graph node, and its SummarizationJobs.status='completed' row (retry_count=13) all still existed. Durable Memory Core tool metrics showed no purge_session call in the 24-hour window containing the recycle.
This is the measured hardening leaf anticipated by #16017: a normal recycle lost one acknowledged derived write.
The Problem
SessionService.summarizeSession() writes session summaries directly to Chroma and then calls verifyPersistedVector(). That verify is intentionally fail-soft and reads from the same live Chroma process. It proves the row/vector is readable now; it does not provide a replayable durability boundary across the supervisor's forced process recycle.
The local supervisor's killTask() uses SIGKILL for Chroma because the daemon ignores SIGTERM. A summary can therefore cross all current success gates, mark its SQLite coordinator row completed, and still vanish when Chroma is recycled seconds later.
#13462 already repaired a different edge: a completed coordinator row no longer permanently suppresses re-summarization when drift detection selects a missing artifact. That remains useful as eventual self-healing, but it is not an acknowledgement contract. The missing row observed here was still absent roughly 18 hours after the recycle, and rebuilding it would rerun the expensive model synthesis instead of replaying the exact acknowledged result.
The Architectural Reality
ai/services/memory-core/SessionService.mjs owns summary synthesis, the deterministic summary_<sessionId> id, the direct Chroma upsert, graph projection, and SummarizationJobs completion.
ai/services/memory-core/helpers/verifyPersistedVector.mjs is a process-local integrity check. Its documented contract is never-throw and no durable retry queue.
ai/graph/storage/SQLite.mjs owns the durable SummarizationJobs coordinator table and is the existing non-Chroma state plane for session-summary work.
ai/daemons/orchestrator/services/ProcessSupervisorService.mjs intentionally sends SIGKILL for Chroma max-runtime recycle. Removing that recycle or assuming graceful Chroma shutdown would move the failure rather than make the receipt honest.
- Raw memories already have a JSONL WAL and embed-daemon path. This ticket should reuse that durability principle, not fold session summaries into the raw-memory WAL or change
add_memory.
The Fix
- Persist a replayable session-summary result envelope in the durable coordinator/SQLite plane before the Chroma write can become an acknowledged completion. The envelope must contain the deterministic id, summary document, and metadata required to idempotently reconstruct the exact Chroma row without another model invocation.
- Keep the current Chroma upsert and vector-integrity check, but make
SummarizationJobs.status='completed' and the sweep's processed count mean: the Chroma row exists and an interrupted write remains recoverable from durable local state.
- On Memory Core/summarizer recovery, replay any staged or completed-but-Chroma-missing envelope idempotently, verify the reconstructed row, then retire or compact the replay payload according to a bounded policy.
- Preserve #13462's drift-based re-synthesis as a fallback for legacy rows that predate the replay envelope. New acknowledged results must use replay, not expensive re-synthesis, after a recycle.
- Prove the boundary with a disposable Chroma instance: acknowledge a summary, terminate Chroma at the observed post-ack seam, restart it, run recovery, and assert the exact document/metadata row is present without invoking the summary model again.
No new environment variable or operator setting is needed; this is an internal durability contract.
Contract Ledger Matrix
| Target Surface |
Source of Authority |
Proposed Behavior |
Fallback |
Docs |
Evidence |
summarizeSession() result |
SessionService.mjs |
A returned result is recoverable across a forced Chroma recycle |
Fail the job before acknowledgement if the durable envelope cannot be written |
Method JSDoc |
Unit mutation plus disposable-Chroma restart witness |
SummarizationJobs.status='completed' |
SQLite coordinator table |
Completed means Chroma-present or replayable from an exact durable result |
#13462 drift repair for legacy rows without an envelope |
Schema/migration JSDoc |
Completed-with-missing-row replay spec |
| Summary replay envelope |
SQLite coordinator/adjacent durable state |
Deterministic, idempotent reconstruction of document + metadata |
Leave pending and fail loud; never discard the only replay copy |
Storage helper/schema docs |
Crash-seam and idempotent replay coverage |
Summary sweep processed count |
summarizeSessions() |
Counts only durability-qualified results |
Failed/staged rows are reported separately |
Existing lifecycle log docs/comments |
Focused lifecycle assertion |
Decision Record impact
none — this repairs the existing Memory Core summary/coordinator contract. It does not change public MCP APIs, cloud deployment ownership, or AiConfig authority.
Acceptance Criteria
Out of Scope
- Reopening the supervisor health-loop repair from #16022.
- Removing the local Chroma max-runtime recycle or changing its SIGKILL behavior.
- Generalizing this one measured summary seam into a durability rewrite for every Chroma collection.
- Raw-memory/message WAL changes,
add_memory, cloud/container deployment, backup/restore policy, or restoring the missing historical row.
- Closing #16017 before this hardening leaf has restart evidence.
Avoided Traps
- Another same-process read-back. It can pass and still lose the row at the next forced recycle; that is the observed failure.
- Sleep before SIGKILL. A timing delay reduces probability but does not create a replayable acknowledgement boundary.
- Rerun the model after every miss. #13462 makes that possible, but it is expensive, can starve older rows, and does not reproduce the exact acknowledged result.
- Put summaries into the raw-memory WAL. The durability principle is shared; the payload, consumer, and lifecycle are not.
- Disable normal recycling. That trades the measured data-loss seam for unbounded daemon lifetime without repairing receipt semantics.
Duplicate / Claim Sweep
Live latest-open sweep: checked the latest 20 open issues immediately before creation; no equivalent found.
A2A in-flight sweep: checked the latest 30 messages across read states immediately before creation; no competing summary-durability claim found.
Targeted GitHub/repository/Knowledge Base sweeps found #16017 (parent incident), #13462 (completed-job drift repair), #14256 (same-process vector read-back), and #12838 (raw-memory WAL) as adjacent prior art, not duplicates.
Related
Related: #16017
Related: #16022
Related: #13462
Related: #14256
Related: #12838
Origin Session ID: 019fa530-53d6-7271-bf05-51497720b29c
Handoff Retrieval Hints:
query_raw_memories("#16017 normal Chroma max-runtime recycle acknowledged session summary missing")
- Exact missing row:
summary_40884abd-711b-4ffd-8349-01a71704bd82
- Exact recycle window:
2026-07-27T22:52:01.786Z..2026-07-27T22:52:14.596Z
Authored by Euclid (GPT-5 Codex, Codex Desktop). Session 019fa530-53d6-7271-bf05-51497720b29c.
Context
The final post-loop durability AC on #16017 has now failed against a natural production event, without restarting shared infrastructure for the test.
On 2026-07-27, the orchestrator completed a five-session summary sweep immediately before the local Chroma daemon reached its normal 24-hour max-runtime recycle:
40884abd-711b-4ffd-8349-01a71704bd82logged3/5 done22:52:01.786processed=5and completed successfully22:52:09.005max-runtime:87281827ms>86400000ms22:52:10.95422:52:14.596processed=5is load-bearing evidence:SessionService.summarizeSessions()only adds a session toprocessedwhensummarizeSession()returns its result object. That return occurs after the deterministic Chroma upsert, the immediate vector read-back, and the SQLite graph upsert.At the 2026-07-28 read-back, four of those five deterministic
summary_<sessionId>rows still existed.summary_40884abd-711b-4ffd-8349-01a71704bd82did not. Its two source memories, itsSESSION_SUMMARYSQLite graph node, and itsSummarizationJobs.status='completed'row (retry_count=13) all still existed. Durable Memory Core tool metrics showed nopurge_sessioncall in the 24-hour window containing the recycle.This is the measured hardening leaf anticipated by #16017: a normal recycle lost one acknowledged derived write.
The Problem
SessionService.summarizeSession()writes session summaries directly to Chroma and then callsverifyPersistedVector(). That verify is intentionally fail-soft and reads from the same live Chroma process. It proves the row/vector is readable now; it does not provide a replayable durability boundary across the supervisor's forced process recycle.The local supervisor's
killTask()uses SIGKILL for Chroma because the daemon ignores SIGTERM. A summary can therefore cross all current success gates, mark its SQLite coordinator rowcompleted, and still vanish when Chroma is recycled seconds later.#13462 already repaired a different edge: a completed coordinator row no longer permanently suppresses re-summarization when drift detection selects a missing artifact. That remains useful as eventual self-healing, but it is not an acknowledgement contract. The missing row observed here was still absent roughly 18 hours after the recycle, and rebuilding it would rerun the expensive model synthesis instead of replaying the exact acknowledged result.
The Architectural Reality
ai/services/memory-core/SessionService.mjsowns summary synthesis, the deterministicsummary_<sessionId>id, the direct Chroma upsert, graph projection, andSummarizationJobscompletion.ai/services/memory-core/helpers/verifyPersistedVector.mjsis a process-local integrity check. Its documented contract is never-throw and no durable retry queue.ai/graph/storage/SQLite.mjsowns the durableSummarizationJobscoordinator table and is the existing non-Chroma state plane for session-summary work.ai/daemons/orchestrator/services/ProcessSupervisorService.mjsintentionally sends SIGKILL for Chroma max-runtime recycle. Removing that recycle or assuming graceful Chroma shutdown would move the failure rather than make the receipt honest.add_memory.The Fix
SummarizationJobs.status='completed'and the sweep'sprocessedcount mean: the Chroma row exists and an interrupted write remains recoverable from durable local state.No new environment variable or operator setting is needed; this is an internal durability contract.
Contract Ledger Matrix
summarizeSession()resultSessionService.mjsSummarizationJobs.status='completed'processedcountsummarizeSessions()Decision Record impact
none— this repairs the existing Memory Core summary/coordinator contract. It does not change public MCP APIs, cloud deployment ownership, or AiConfig authority.Acceptance Criteria
processed/completedreceipt.add_memorybehavior, or cloud/Compose ownership change is introduced.Out of Scope
add_memory, cloud/container deployment, backup/restore policy, or restoring the missing historical row.Avoided Traps
Duplicate / Claim Sweep
Live latest-open sweep: checked the latest 20 open issues immediately before creation; no equivalent found.
A2A in-flight sweep: checked the latest 30 messages across read states immediately before creation; no competing summary-durability claim found.
Targeted GitHub/repository/Knowledge Base sweeps found #16017 (parent incident), #13462 (completed-job drift repair), #14256 (same-process vector read-back), and #12838 (raw-memory WAL) as adjacent prior art, not duplicates.
Related
Related: #16017 Related: #16022 Related: #13462 Related: #14256 Related: #12838
Origin Session ID: 019fa530-53d6-7271-bf05-51497720b29c
Handoff Retrieval Hints:
query_raw_memories("#16017 normal Chroma max-runtime recycle acknowledged session summary missing")summary_40884abd-711b-4ffd-8349-01a71704bd822026-07-27T22:52:01.786Z..2026-07-27T22:52:14.596ZAuthored by Euclid (GPT-5 Codex, Codex Desktop). Session 019fa530-53d6-7271-bf05-51497720b29c.