LearnNewsExamplesServices
Frontmatter
id13974
titleExpose REM extractor failure terminality to DreamService
stateClosed
labels
enhancementaitestingarchitectureperformancemodel-experience
assigneesneo-gpt
createdAtJun 25, 2026, 12:10 AM
updatedAtJun 25, 2026, 12:55 AM
githubUrlhttps://github.com/neomjs/neo/issues/13974
authorneo-gpt
commentsCount0
parentIssue13835
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtJun 25, 2026, 12:55 AM

Expose REM extractor failure terminality to DreamService

Closed v13.1.0/archive-v13-1-0-chunk-6 enhancementaitestingarchitectureperformancemodel-experience
neo-gpt
neo-gpt commented on Jun 25, 2026, 12:10 AM

Context

#13835's first tri-state kernel shipped the steady-cadence bleed stop for deterministic skip-over-band, but current dev still collapses every SemanticGraphExtractor.executeTriVectorExtraction() failure to bare null. DreamService.processUndigestedSessions() therefore guesses deferReason from payload size and cannot distinguish a provider guardrail skip, a length-truncated completion, silent empty response, repair-prompt overflow, schema exhaustion, timeout, or provider throw.

V-B-A evidence checked 2026-06-24T22:09:42Z:

  • ai/services/graph/SemanticGraphExtractor.mjs returns null at all failure exits.
  • ai/daemons/orchestrator/services/DreamService.mjs reads success ? ... : 'null-result' and derives deferReason from payloadSizeTokens plus aiConfig.localModels?.chat?.safeProcessingLimitTokens.
  • #13835 comment https://github.com/neomjs/neo/issues/13835#issuecomment-4770152896 records the typed-extractor-contract follow-up as the remaining bleed-stop refinement.

The Problem

The REM cadence needs terminality evidence from the component that actually saw the provider failure. Without a typed extractor failure contract, DreamService has to infer terminality from the session payload after the fact. That produces two bad outcomes:

  1. deterministic terminal failures are only partly visible to the cadence state-machine;
  2. retryable failures and terminal failures share the same null-result run-state shape.

This keeps #13835 open even though the tri-state metadata shell exists.

The Architectural Reality

  • SemanticGraphExtractor.executeTriVectorExtraction() owns provider invocation, guardrail friction, finish-reason handling, empty-response detection, repair-loop sizing, schema exhaustion, and provider thrown-error classification.
  • DreamService.processUndigestedSessions() owns digest metadata (digestState, digestAttempts, deferReason) and must not reclassify provider-level evidence by payload-size guessing when the extractor can return the typed evidence directly.
  • ADR 0019 applies because the DreamService read site currently uses a defensive ?. config read; the fix must read resolved AiConfig leaves directly at the use site and fail loud.

The Fix

Add a typed failure descriptor for the extractor failure path while preserving the success payload shape:

  • success remains the parsed Tri-Vector payload;
  • failure returns an envelope such as {ok:false, deferReason, terminalForCadence, frictionSymptom, evidence};
  • DreamService detects ok === false, records the descriptor in per-session telemetry, persists deferReason, and defers only when terminalForCadence === true and digestAttempts >= maxDigestAttempts;
  • remove the payload-size heuristic and the defensive optional-chain config read at the DreamService site.

Keep ConsumerFriction V1 stable: do not add under-band-choke to VALID_SYMPTOMS; persist cadence-specific deferReason separately.

Contract Ledger Matrix

Target Surface Source of Authority Proposed Behavior Fallback Docs Evidence
SemanticGraphExtractor.executeTriVectorExtraction() failure return #13835 and current extractor call sites returns typed failure descriptor instead of bare null on failure; success payload unchanged existing callers treating only truthiness must be updated in-scope JSDoc update unit tests for guardrail, empty response, schema exhaustion
DreamService digest metadata #13835 tri-state state-machine uses extractor deferReason / terminalForCadence instead of payload-size guessing ingest errors remain retryable ingestion-failure inline JSDoc/comments DreamService unit tests
ConsumerFriction symptom enum consumerFrictionHelper.mjs V1 no enum expansion; typed cadence reason is separate existing context-overflow, size-precheck-skip, parse-failure, timeout remain no docs change unless test names need updating existing helper tests stay valid

Decision Record impact

Aligned with ADR 0019: read AiConfig leaves directly at the use site, no defensive optional chaining, no hidden defaults. Aligned with ADR 0023 / #13835 consolidation-liveness: bound un-digestible sessions without claiming deep extraction.

Acceptance Criteria

  • SemanticGraphExtractor.executeTriVectorExtraction() returns typed failure descriptors for guardrail skips, provider length truncation, empty-response choke, repair-prompt overflow, schema exhaustion, and caught provider failures.
  • Successful extraction callers still receive the existing Tri-Vector payload shape.
  • DreamService.processUndigestedSessions() consumes typed descriptors and removes the payload-size deferReason heuristic.
  • terminalForCadence controls deferred marking after maxDigestAttempts; retryable descriptor failures stay undigested.
  • DreamService per-session telemetry records descriptor details instead of collapsing every extractor failure to null-result.
  • Unit coverage updates extractor and DreamService specs for the typed contract.

Out of Scope

  • Deep extraction or chunk-and-reduce of deferred sessions (#12439 / #12073 scope).
  • Expanding consumerFrictionHelper.VALID_SYMPTOMS.
  • Changing model provider readiness or local-model residency behavior.

Related

Parent: #13835. Prior kernel: PR #13843. Guardrail hardening: #13918 / PR #13919 / PR #13922.

Live latest-open sweep: checked latest 20 open GitHub issues at 2026-06-24T22:09:42Z; no equivalent sub-leaf found. A2A in-flight sweep: checked latest 30 all-state messages at 2026-06-24T22:09:42Z; no competing typed-contract claim found. KB duplicate sweep: typed extractor failure contract under-band-choke deferReason terminalForCadence SemanticGraphExtractor DreamService returned #13835 as the top hit and no duplicate leaf.

Release classification: boardless — post-kernel REM hardening under an existing open Agent OS backlog ticket.

Retrieval Hint: typed extractor failure contract terminalForCadence deferReason SemanticGraphExtractor DreamService #13835

tobiu referenced in commit c97801c - "fix(ai): expose REM extractor failure terminality (#13974) (#13975) on Jun 25, 2026, 12:55 AM
tobiu closed this issue on Jun 25, 2026, 12:55 AM