LearnNewsExamplesServices
Frontmatter
id16449
titleThe lifecycle audit reads a stale snapshot without saying so
stateClosed
labels
bugaiarchitecture
assigneesneo-opus-ada
createdAtAug 3, 2026, 5:35 PM
updatedAtAug 5, 2026, 1:23 PM
githubUrlhttps://github.com/neomjs/neo/issues/16449
authorneo-opus-grace
commentsCount1
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtAug 5, 2026, 1:23 PM

The lifecycle audit reads a stale snapshot without saying so

Closed Backlog/active-chunk-12 bugaiarchitecture
neo-opus-grace
neo-opus-grace commented on Aug 3, 2026, 5:35 PM

Context

Found 2026-08-03 while closing D#16304 after its graduation. I had recorded RESOLVED in a comment and never closed the Discussion; npm run ai:audit-discussion-lifecycle caught it. Running the guard for the first time in this session surfaced 41 candidates — so the miss is systemic, not mine alone.

The Problem

Three defects stack, and each one hides the next.

1. The guard reaches no surface

npm run ai:audit-discussion-lifecycle is read-only by design — correct — but nothing invokes it. No workflow, no pre-commit hook, no scheduled job, no skill step. It is a command that must be remembered, and it is not.

Current census, from a live run:

candidates: 41 (graduated-open=6, resolved-only-review=8, stale-open=27)

Six Discussions carried [GRADUATED_TO_TICKET] and stayed open — ages 15 to 101 days. A graduated-but-open Discussion is a context-window bloat trap: it stays in search paths and reads as live ideation, so every future reader pays for a resolved thread.

2. Graduation records the ticket and skips the close

ideation-sandbox-workflow.md §6.7 states the sequence — file the artifact → record [GRADUATED_TO_TICKET: #N] → close RESOLVED. The first two steps produce visible artifacts; the third produces nothing an author sees. So it is the step that gets dropped, and nothing catches it.

One of the six had graduated in proseGRADUATED to Epic #10291, ratified — with no canonical marker at all, so it was not machine-readable and no audit that keys on the marker could have found it either.

3. The guard's input is a lagging snapshot, and its refresh is currently broken

audit-discussion-lifecycle.mjs:20 reads resources/content/discussions — a local ingested snapshot, not live GitHub:

const DEFAULT_DISCUSSIONS_DIR = path.resolve(process.cwd(), 'resources/content/discussions');

After closing all six Discussions on GitHub, the guard still reports all six as graduated-open, because the snapshot has not re-ingested. And the data-sync pipeline that would refresh it is failing — #16428, five consecutive failures.

So the guard currently emits stale verdicts as current findings. A maintainer acting on today's output would re-close six already-closed Discussions, conclude the tool is wrong, and stop trusting it — which is worse than the tool not existing, and is the same false-confidence class as a green check on an unmeasured axis.

The Architectural Reality

  • ai/scripts/diagnostics/audit-discussion-lifecycle.mjs — the guard; snapshot-sourced at :20, read-only by design (correctly: it must never close Discussions or edit bodies unilaterally).
  • .agents/skills/ideation-sandbox/audits/discussion-lifecycle-closure.md — the trigger matrix. Sound; nothing routes to it.
  • .agents/skills/ideation-sandbox/references/ideation-sandbox-workflow.md §6.7 — names closure as the author's final act, with no mechanical follow-through.
  • #16428 — the data-sync pipeline failing, which is what makes defect 3 live rather than theoretical.

Not in question: the guard's read-only design. A tool that closed Discussions autonomously would be worse. The gap is that its findings reach nobody, and that its input can silently lag.

The Fix

  1. Route the guard to a surface. A scheduled workflow, or an extension of an existing maintenance lane, emitting the graduated-open set where a maintainer sees it. graduated-open is unambiguous and mechanical; stale-open and resolved-only-review stay advisory by design.
  2. Make the closure step visible at graduation time. §6.7's third step should produce a check the author can fail — the same way a PR body lint fails on a missing anchor — rather than relying on memory.
  3. Make snapshot staleness legible. The guard should report the snapshot's ingest timestamp with its verdict, and say plainly when that timestamp is older than some bound, so a stale verdict cannot masquerade as a current one. Reading live GitHub is an alternative; carrying the freshness in the output is the cheaper fix and is honest either way.

Contract Ledger Matrix

Target Surface Source of Authority Proposed Behavior Fallback / Error Semantics Docs Evidence
lifecycle-audit invocation surface this ticket graduated-open findings reach a maintainer without anyone remembering the command Surface unavailable ⇒ the audit still runs and exits non-zero locally; it must never close anything itself workflow + skill note a graduated-but-open Discussion is reported without a human running the command
guard output freshness audit-discussion-lifecycle.mjs Every verdict carries the snapshot's ingest time; a stale snapshot is stated, never implied Snapshot missing/unreadable ⇒ fail loudly rather than report zero candidates guard JSDoc closing a Discussion and re-running reports either the change or an explicit staleness notice — never a silent false positive
§6.7 closure step ideation-sandbox-workflow.md Recording [GRADUATED_TO_TICKET] without closing is detectable at graduation time Detection absent ⇒ status quo (the audit remains the backstop) workflow §6.7 a graduation that records the marker and leaves the Discussion open is caught

Decision Record impact

none — routing and observability over an existing audit. No change to graduation semantics or to the guard's read-only contract.

Acceptance Criteria

SCOPE NARROWED 2026-08-04. This ticket bundled TWO independent defects at filing — "the audit reaches nobody" and "reads a stale snapshot." Only the snapshot half is delivered here. The two ACs below were moved verbatim to #16524, not dropped:

  • graduated-open findings reach a maintainer-visible surface without anyone invoking the command manually.
  • Recording [GRADUATED_TO_TICKET] while leaving the Discussion open is detectable at graduation time, not only by a later audit.

Splitting rather than stretching the close target: a PR delivering four of six ACs cannot honestly say Resolves, and silently narrowing the ticket to fit the diff would erase the undelivered work instead of tracking it. The ordering is also load-bearing — automating the guard before it could state its own staleness would have broadcast stale findings on a cron, which is worse than the silence it replaces.

  • The guard's output states the snapshot's ingest timestamp, and explicitly flags a snapshot older than a stated bound.
  • A missing or unreadable snapshot fails loudly rather than reporting zero candidates.
  • The guard remains read-only: it never closes a Discussion, edits a body, or posts a comment.
  • stale-open and resolved-only-review remain advisory and are not escalated into mechanical failures — they require judgement by design.

Out of Scope

  • Auto-closing Discussions. The guard's read-only design is correct and this ticket does not change it.
  • The 27 stale-open and 8 resolved-only-review candidates. They need maintainer judgement per the trigger matrix; this ticket is about the mechanical graduated-open class and the guard's reachability.
  • Fixing the data-sync pipeline. #16428 owns that. This ticket makes its failure visible in the guard's output rather than silently corrupting the verdict.

Avoided Traps

  • Making the guard close things. It would then be a bot mutating peer-owned substrate on a heuristic, and the trigger matrix explicitly reserves stale-open for review.
  • Treating this as author discipline. Six misses across 15–101 days and multiple authors is a substrate problem. "Remember step three" has already failed at scale, and I contributed one of the misses today with the workflow open in front of me.
  • Trusting the snapshot silently. The current output is confidently wrong about six Discussions right now. A guard that cannot say how fresh its input is will eventually be disbelieved, and then it protects nothing.

Related

  • #16428 — data-sync pipeline failures; why the snapshot cannot currently refresh.
  • D#16304 → #16448 — the graduation whose closure miss surfaced this.
  • .agents/skills/ideation-sandbox/audits/discussion-lifecycle-closure.md — the trigger matrix this routes.

Live latest-open sweep: checked latest open issues at 2026-08-03T15:34Z; no equivalent found.

Origin Session ID: 9f05cd72-5457-4ec2-926c-ef1406041f19

Retrieval Hint: query_raw_memories("discussion lifecycle audit graduated-open never closed snapshot stale read-only guard reaches no surface")

tobiu referenced in commit 0f69994 - "fix(ai): the lifecycle audit states how old its snapshot is (#16449) (#16525) on Aug 5, 2026, 1:23 PM
tobiu closed this issue on Aug 5, 2026, 1:23 PM