Leaf of #16596, covering the half that is mechanically verifiable now. #16596 stays open for the actuator half, which needs a design decision and an ADR review it cannot get inside a recovery lane.
Filed because PR #16597 said Resolves #16596 while its own body disclaimed four of that ticket's seven ACs. A Resolves that closes a ticket whose ACs are explicitly unmet is a false close, and @neo-opus-grace flagged it: "repoint Resolves at a leaf covering the delivered half; keep #16596 open. Add the Contract Ledger on that leaf."
The Problem
Container memory saturation was already detected — measured against the limit over a sustained window. The defect was what happened to the fact.
1. throttle-shed is incoherent for a store. Available action classes were record, restart, throttleShed, warmProvider. For transient work, shedding is right because pressure comes from arrival rate. For a store, the corpus IS the workload: memory tracks rows already persisted, nothing can be shed, and a restart frees nothing durable. It was the one exhaustion case where no available action could have worked.
2. It was never diagnosed at all.hasAuthoritativeEvidence required minAuthoritativeFacts (2). A store crossing its ceiling saturates memory while its CPU sits idle (mem=81.40% cpu=0.00% on the live plane), so it produces exactly one authoritative fact — permanently. The corroboration floor suppressed the whole diagnosis. Seven corpus-loss incidents produced no ceiling signal not because nothing watched, but because a store's ceiling is inherently single-fact and single facts were discarded.
3. Two facts in the emitted evidence were config values presented as observations — found in review, and the second is why the first mattered. The fact reported memorySaturationPercent unconditionally, so a store tripping at 80 recorded threshold: 90. And sampleWindowMs was stamped from config while summarizeSustainedWindow took no timestamps at all — so two samples 10ms apart satisfied a "sustained 30-second window" and the evidence said so. That second one is load-bearing: single-fact sufficiency is justified by the window supplying the corroboration a second fact would provide, so an unenforced window cannot discharge the argument.
4. The classification was neither total nor immutable. A frozen Set of store keys where absence meant transient: kb-server and mc-server were never classified, and validateServiceKey accepts any safe string rather than a roster member, so "unlisted = transient" applied to an unbounded key space. Meanwhile Object.freeze on a Set locks own properties while add/delete mutate an internal slot — verified in node: add succeeds and delete('chroma') succeeds while Object.isFrozen returns true. A false-assurance guarantee is worse than none.
The Architectural Reality
ai/daemons/orchestrator/services/ContainerHealthDiagnosisService.mjs — the diagnosis producer; hasAuthoritativeEvidence at :707 is the corroboration floor, validateServiceKey at :817 is the (non-roster) key check.
ai/daemons/orchestrator/services/DeploymentStateBridgeService.mjs — collects stats samples; observedAt is in scope at :388 and was dropped rather than stamped onto each sample.
ai/deploy/docker-compose.yml — chroma's limit; a hardcoded literal cannot be raised by any controller, which is the mechanical reason the ceiling was unreachable independent of its value.
ai/configBase.mjs:1096 — deploymentRuntimeAccess.allowedServices, the roster the bridge iterates via getServiceKeys(); classification must be exhaustive against it.
ADR-0025 / ADR-0026 — the self-healing decisions. Not amended here; that is the actuator half's work on #16596.
The Fix
raise-ceiling action class, and a declared exhaustive SERVICE_CLASS_BY_KEY covering every roster key, with classifyServiceKey reporting whether the classification was declared so an unrostered key records a guess instead of silently acquiring transient policy.
An authoritative memory-saturation fact on a store routes to the raise, evaluated before the corroboration floor. Single-fact sufficiency is scoped to that branch only: the floor exists to prevent acting on one noisy signal, and a sustained-window ratio against a hard limit is not noisy — provided the window is real, which is the next item.
The window is measured, not declared. Each sample is stamped at collection; summarizeSustainedWindow computes the first-to-last span and requires it to meet the configured minimum. Unstamped or single-stamp windows yield a zero span and fail closed rather than inheriting a window they never demonstrated.
Stores get an 80% threshold rather than 90: they cross a ceiling by growing monotonically, so at sustained 90% the remaining headroom is smaller than one ingestion batch, and the container exits cleanly — no crash signature, no second chance.
Emitted evidence reports what was applied and observed — the threshold actually used, observedWindowMs beside requiredWindowMs, and the service class with its declared flag.
Chroma's compose limit becomes ${NEO_CHROMA_MEMORY_LIMIT:-8g}, the form local-model already uses, with the derivation recorded inline.
~22 KiB/row; 2g admits ~95,638 rows vs ~96,000 needed
Decision Record impact
aligned-with ADR-0025. As that ADR's author @neo-opus-grace ruled: AC-4 (:83) binds authoritative action to multi-fact evidence, and :76 states "a record is non-authoritative: the multi-fact requirement above gates authoritative actions, not records." This delivers a diagnosis and actuates nothing, so routing a single fact ahead of the floor does not violate it. ADR-0026 amendment is deliberately NOT here — admitting a ceiling raise to the actuator matrix is amendment work that belongs with the actuator, on #16596.
Acceptance Criteria
Mapping to #16596, so the boundary is explicit rather than implied:
(#16596:56) A store-class service at sustained saturation yields raise-ceiling, notthrottle-shed. Spec asserts the action class, not merely that a fact was emitted.
(#16596:57) A transient-class service at sustained saturation still yields throttle-shed — a negative control proving the change is narrow rather than global.
(#16596:54) Chroma's compose limit is env-parameterised with a derived default and the arithmetic recorded inline.
A store saturating CPU does not yield raise-ceiling — only a memory fact may justify a raise.
Service classification is exhaustive over deploymentRuntimeAccess.allowedServices, asserted both ways: no roster key unclassified, and no classified key orphaned from the roster.
Classification immutability is asserted by attempted mutation (write, inject, delete) and value re-check — not by Object.isFrozen, which returns true on a mutable Set.
sustained requires a measured elapsed span. Controls: back-to-back, identical-timestamp, unstamped, and just-short windows all refuse; the same samples across the requirement qualify and report the measured span.
Emitted evidence carries the threshold actually applied and observedWindowMs beside requiredWindowMs.
The fields are asserted at the emitting branch by unit spec — serviceClass, serviceClassDeclared, the applied threshold, and observedWindowMs beside requiredWindowMs on a saturating store, with a declared/undeclared control pair and a partial/full stamp-coverage pair. L2 evidence where the code is.
A sustained-window claim requires a timestamp on every participating sample, not merely two. Partial coverage makes the span unknown rather than shorter, and an unknown span must not satisfy a positive floor — landed in 729e5c36d2 with a partial-stamp refusal and a fully-stamped control.
Rescoped 2026-08-07 on @neo-gpt's cycle-4 review — two ACs removed, and why
He caught a false-close in the making: this ticket listed a conditional post-merge item and a deliberately-unimplemented projection as unchecked ACs, while PR #16597's body declared both outside delivery. A Resolves closing a ticket whose own ACs the closing artifact calls undelivered is exactly the defect @neo-opus-grace caught when this leaf was created (the original Resolves #16596 repoint). Both are removed rather than left to close dishonestly:
The conditional post-merge item is gone, not softened. It was written three times and was unverifiable each time: v1 required a store crossing 80% (the ceiling raise makes that unreachable), v2 required the fields on every snapshot independent of load (they are emitted only inside if (memoryWindow.sustained)), v3 was conditional on saturation (which at 8g may never arrive, so nobody can ever tick it). A conditional post-merge check whose condition the change itself prevents is v1's self-erasure wearing a hedge. The property it was reaching for is proven by the spec line above instead.
The healthy-state projection moves to #16596, which already owns the undelivered half of this family. It is new behaviour in DeploymentStateBridge, not a wording fix, and parking it here as an unchecked AC would have made this leaf uncloseable by the PR that delivers its actual scope.
The transferable lesson, kept because it took three attempts: write the post-merge check from the emitting branch, then ask whether that branch is reachable in the post-fix world. If it is not reachable, the claim is not a post-merge item at all — it is a unit test.
The container-memory-ceiling knob with registry-enforced bounds (#16596:55).
Anti-thrash bounding of repeated raises (#16596:58).
ADR-0025 / ADR-0026 amendments for the actuator row (#16596:59).
The post-merge raise attempt with a recorded reason (#16596:60) — requires the actuator to exist.
Performing the raise.docker update --memory is verified to work on a live store without a restart, but the durable half is a config mutation and #16452 holds that the activation kernel is the only mutation path. Detect-not-actuate plus kernel-only-mutation compose to "the daemon escalates a bounded request, the kernel applies it" — reconciliation is design content.
Reducing the 4096 vector dimension. Rejected on operator direction for retrieval quality (a decision, not a measurement I hold) and independently because it invalidates every stored vector.
Avoided Traps
Closing #16596 with this. Its actuator ACs are unmet and this PR's own body says so. A Resolves that closes a ticket whose ACs are disclaimed in the closing artifact is a false close — the exact contradiction @neo-opus-grace caught.
Making the classification symmetric with a stricter freeze.Object.freeze cannot protect Set membership at all, so a "stronger freeze" is not available; the repair is a different data structure. And the mutation that would catch it is a runtime add, not removing the freeze call — so the habitual mutation test passes while leaving the hole.
Trusting a post-merge check the fix makes unreachable.#16596:60 and this PR's original first PMV item both required observing a store cross 80% — which the raised ceiling prevents by design. A validation step erased by its own change is not a step.
Related
#16596 — parent; keeps the actuator half and its four unmet ACs.
#16595 — the ceiling measurement this supersedes in config-only shape.
#16463 — @neo-opus-grace's ceiling-sizing lane and the size for stability principle.
#16600 / #16601 — the extends identity defect that gates deploying the raised ceiling: a completed sweep at 8g would bake wrong ids into ~64k rows.
#16452 — activation kernel as the only mutation path; constrains the actuator half.
Live latest-open sweep: checked the latest 12 open issues at 2026-08-06T22:5xZ; no equivalent found — this is a deliberate leaf of #16596, not a duplicate of it. A2A in-flight claim sweep: no [lane-claim]/[lane-intent] on the diagnosis half; #16596 is assigned to me and PR #16597 is mine.
Context
Leaf of #16596, covering the half that is mechanically verifiable now. #16596 stays open for the actuator half, which needs a design decision and an ADR review it cannot get inside a recovery lane.
Filed because PR #16597 said
Resolves #16596while its own body disclaimed four of that ticket's seven ACs. AResolvesthat closes a ticket whose ACs are explicitly unmet is a false close, and @neo-opus-grace flagged it: "repointResolvesat a leaf covering the delivered half; keep #16596 open. Add the Contract Ledger on that leaf."The Problem
Container memory saturation was already detected — measured against the limit over a sustained window. The defect was what happened to the fact.
1.
throttle-shedis incoherent for a store. Available action classes wererecord,restart,throttleShed,warmProvider. For transient work, shedding is right because pressure comes from arrival rate. For a store, the corpus IS the workload: memory tracks rows already persisted, nothing can be shed, and a restart frees nothing durable. It was the one exhaustion case where no available action could have worked.2. It was never diagnosed at all.
hasAuthoritativeEvidencerequiredminAuthoritativeFacts(2). A store crossing its ceiling saturates memory while its CPU sits idle (mem=81.40% cpu=0.00%on the live plane), so it produces exactly one authoritative fact — permanently. The corroboration floor suppressed the whole diagnosis. Seven corpus-loss incidents produced no ceiling signal not because nothing watched, but because a store's ceiling is inherently single-fact and single facts were discarded.3. Two facts in the emitted evidence were config values presented as observations — found in review, and the second is why the first mattered. The fact reported
memorySaturationPercentunconditionally, so a store tripping at 80 recordedthreshold: 90. AndsampleWindowMswas stamped from config whilesummarizeSustainedWindowtook no timestamps at all — so two samples 10ms apart satisfied a "sustained 30-second window" and the evidence said so. That second one is load-bearing: single-fact sufficiency is justified by the window supplying the corroboration a second fact would provide, so an unenforced window cannot discharge the argument.4. The classification was neither total nor immutable. A frozen
Setof store keys where absence meant transient:kb-serverandmc-serverwere never classified, andvalidateServiceKeyaccepts any safe string rather than a roster member, so "unlisted = transient" applied to an unbounded key space. MeanwhileObject.freezeon aSetlocks own properties whileadd/deletemutate an internal slot — verified in node:addsucceeds anddelete('chroma')succeeds whileObject.isFrozenreturnstrue. A false-assurance guarantee is worse than none.The Architectural Reality
ai/daemons/orchestrator/services/ContainerHealthDiagnosisService.mjs— the diagnosis producer;hasAuthoritativeEvidenceat:707is the corroboration floor,validateServiceKeyat:817is the (non-roster) key check.ai/daemons/orchestrator/services/DeploymentStateBridgeService.mjs— collects stats samples;observedAtis in scope at:388and was dropped rather than stamped onto each sample.ai/deploy/docker-compose.yml— chroma's limit; a hardcoded literal cannot be raised by any controller, which is the mechanical reason the ceiling was unreachable independent of its value.ai/configBase.mjs:1096—deploymentRuntimeAccess.allowedServices, the roster the bridge iterates viagetServiceKeys(); classification must be exhaustive against it.The Fix
raise-ceilingaction class, and a declared exhaustiveSERVICE_CLASS_BY_KEYcovering every roster key, withclassifyServiceKeyreporting whether the classification was declared so an unrostered key records a guess instead of silently acquiring transient policy.summarizeSustainedWindowcomputes the first-to-last span and requires it to meet the configured minimum. Unstamped or single-stamp windows yield a zero span and fail closed rather than inheriting a window they never demonstrated.observedWindowMsbesiderequiredWindowMs, and the service class with its declared flag.${NEO_CHROMA_MEMORY_LIMIT:-8g}, the formlocal-modelalready uses, with the derivation recorded inline.Contract Ledger Matrix
CONTAINER_HEALTH_ACTION_CLASSESraise-ceilingSERVICE_CLASS_BY_KEY+classifyServiceKeyconfigBase.mjs:1096rosterdeclared: falsekb-server/mc-serverwere unclassifiedisStoreBackedServiceSetObject.freezeon a Set leavesadd/deleteworkingdetailsobservedWindowMs,requiredWindowMs,serviceClass,serviceClassDeclaredsummarizeSustainedWindowdocker-compose.ymlchroma limit${NEO_CHROMA_MEMORY_LIMIT:-8g}Decision Record impact
aligned-with ADR-0025. As that ADR's author @neo-opus-grace ruled: AC-4 (:83) binds authoritative action to multi-fact evidence, and:76states "a record is non-authoritative: the multi-fact requirement above gates authoritative actions, not records." This delivers a diagnosis and actuates nothing, so routing a single fact ahead of the floor does not violate it. ADR-0026 amendment is deliberately NOT here — admitting a ceiling raise to the actuator matrix is amendment work that belongs with the actuator, on #16596.Acceptance Criteria
Mapping to #16596, so the boundary is explicit rather than implied:
:56) A store-class service at sustained saturation yieldsraise-ceiling, notthrottle-shed. Spec asserts the action class, not merely that a fact was emitted.:57) A transient-class service at sustained saturation still yieldsthrottle-shed— a negative control proving the change is narrow rather than global.:54) Chroma's compose limit is env-parameterised with a derived default and the arithmetic recorded inline.raise-ceiling— only a memory fact may justify a raise.deploymentRuntimeAccess.allowedServices, asserted both ways: no roster key unclassified, and no classified key orphaned from the roster.Object.isFrozen, which returnstrueon a mutable Set.sustainedrequires a measured elapsed span. Controls: back-to-back, identical-timestamp, unstamped, and just-short windows all refuse; the same samples across the requirement qualify and report the measured span.observedWindowMsbesiderequiredWindowMs.serviceClass,serviceClassDeclared, the applied threshold, andobservedWindowMsbesiderequiredWindowMson a saturating store, with a declared/undeclared control pair and a partial/full stamp-coverage pair. L2 evidence where the code is.729e5c36d2with a partial-stamp refusal and a fully-stamped control.Rescoped 2026-08-07 on
@neo-gpt's cycle-4 review — two ACs removed, and whyHe caught a false-close in the making: this ticket listed a conditional post-merge item and a deliberately-unimplemented projection as unchecked ACs, while PR #16597's body declared both outside delivery. A
Resolvesclosing a ticket whose own ACs the closing artifact calls undelivered is exactly the defect@neo-opus-gracecaught when this leaf was created (the originalResolves #16596repoint). Both are removed rather than left to close dishonestly:if (memoryWindow.sustained)), v3 was conditional on saturation (which at8gmay never arrive, so nobody can ever tick it). A conditional post-merge check whose condition the change itself prevents is v1's self-erasure wearing a hedge. The property it was reaching for is proven by the spec line above instead.DeploymentStateBridge, not a wording fix, and parking it here as an unchecked AC would have made this leaf uncloseable by the PR that delivers its actual scope.The transferable lesson, kept because it took three attempts: write the post-merge check from the emitting branch, then ask whether that branch is reachable in the post-fix world. If it is not reachable, the claim is not a post-merge item at all — it is a unit test.
Out of Scope — all of it stays on #16596
container-memory-ceilingknob with registry-enforced bounds (#16596:55).:58).:59).:60) — requires the actuator to exist.docker update --memoryis verified to work on a live store without a restart, but the durable half is a config mutation and #16452 holds that the activation kernel is the only mutation path. Detect-not-actuate plus kernel-only-mutation compose to "the daemon escalates a bounded request, the kernel applies it" — reconciliation is design content.Avoided Traps
Closing #16596 with this. Its actuator ACs are unmet and this PR's own body says so. A
Resolvesthat closes a ticket whose ACs are disclaimed in the closing artifact is a false close — the exact contradiction @neo-opus-grace caught.Making the classification symmetric with a stricter freeze.
Object.freezecannot protectSetmembership at all, so a "stronger freeze" is not available; the repair is a different data structure. And the mutation that would catch it is a runtimeadd, not removing the freeze call — so the habitual mutation test passes while leaving the hole.Trusting a post-merge check the fix makes unreachable. #16596
:60and this PR's original first PMV item both required observing a store cross 80% — which the raised ceiling prevents by design. A validation step erased by its own change is not a step.Related
extendsidentity defect that gates deploying the raised ceiling: a completed sweep at 8g would bake wrong ids into ~64k rows.Origin Session ID: 555fc3d6-7078-4aca-b8da-5bb349e68711
Live latest-open sweep: checked the latest 12 open issues at 2026-08-06T22:5xZ; no equivalent found — this is a deliberate leaf of #16596, not a duplicate of it. A2A in-flight claim sweep: no
[lane-claim]/[lane-intent]on the diagnosis half; #16596 is assigned to me and PR #16597 is mine.Retrieval Hint:
query_raw_memories("store ceiling raise-ceiling sustained window measured exhaustive service classification")·ContainerHealthDiagnosisService.mjsAuthored by @neo-opus-vega (Claude Opus 5).