LearnNewsExamplesServices
Frontmatter
id15882
titleQueryReRanker asserts over an unscoped shared collection — one false-fail, one unfalsifiable
stateClosed
labels
bugaitesting
assigneesneo-opus-ada
createdAtJul 25, 2026, 1:30 AM
updatedAtJul 25, 2026, 1:57 AM
githubUrlhttps://github.com/neomjs/neo/issues/15882
authorneo-opus-ada
commentsCount0
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtJul 25, 2026, 1:57 AM

QueryReRanker asserts over an unscoped shared collection — one false-fail, one unfalsifiable

Closed Backlog/active-chunk-9 bugaitesting
neo-opus-ada
neo-opus-ada commented on Jul 25, 2026, 1:30 AM

Context

#15874 cohort 2, traced. This is the leaf that fixes it; #15874 remains open for cohort 1, whose mechanism is opposite and unresolved.

QueryReRanker.spec.mjs:584 fails under --workers=4 with unit-brain serialized, and passes wide — the counterintuitive direction that made it worth tracing.

The Problem

The test adds one fixture row, then makes two queries against the run-scoped Chroma collection that every Brain spec writes to. Neither query is scoped to the fixture:

const recentResults = await collection.get({
    where  : {timestamp: {'$gt': thirtyDaysAgo}},   // bounded by time, unbounded in count
    include: ['metadatas']
});
expect(recentSessionIds).not.toContain(ancientSessionId);

const allResults = await collection.get({include: ['metadatas']});   // no where at all
expect(allSessionIds).toContain(ancientSessionId);

Both assertions therefore depend on the total size of a collection the test does not control. That breaks in two opposite ways:

Assertion Failure mode Observed?
toContain (unscoped) FALSE FAIL — once the collection outgrows a returned page, the fixture's own row falls outside it Yes — this is the #15874 cohort-2 failure
not.toContain (time-scoped only) FALSE PASS — a truncated page is less likely to contain the row, so it stays green even if the $gt filter is completely broken Never failed — which is worse

The second one is the more serious defect. It has never gone red, because it cannot: it is unfalsifiable in the presence of truncation. A test asserting "the filter excluded my row" that would also pass if the filter did nothing is not testing the filter.

Why serialization surfaced it

With unit-brain at workers: 1, all Brain specs run sequentially in one worker, so everything written earlier has accumulated by the time this test runs. Wide, the same specs spread across four processes and this one can execute against a smaller collection. Less concurrency means more accumulated state at this point, not less — which is why reducing parallelism broke it.

The Architectural Reality

  • test/playwright/unit/ai/services/memory-core/QueryReRanker.spec.mjs:584
  • The collection comes from SDK.Memory_ChromaManager.getMemoryCollection() — run-scoped, shared by every unit-brain spec.
  • This is a query-shape bug, not an isolation bug. It is wrong at any worker count; parallelism only changed how quickly the collection grew past the threshold.

The Fix

Scope both queries to the fixture's own sessionId, making them size-independent:

  • Negative assertionwhere: {$and: [{sessionId}, {timestamp: {$gt: …}}]}, and additionally assert the scoped result is empty. That converts an unfalsifiable check into a real one: with the filter working the scoped window returns nothing; if it regressed, this row would come back.
  • Positive assertionwhere: {sessionId}, so the fixture's row is retrievable regardless of collection size.

No isolation work, no harness change, no worker-count dependency.

Acceptance Criteria

  • Both queries scoped to the fixture's sessionId
  • The negative assertion is falsifiable — the scoped 30-day window asserted empty, so a broken $gt filter fails the test
  • Passes at --workers=1 (the config that exposed it) and inside a wide run
  • No assertion weakened — the test proves strictly more than before
  • #15874's cohort 2 is closed; cohort 1 explicitly untouched

Out of Scope

  • #15874 cohort 1 (ReceiptDurability, Smoke). Opposite polarity — it fails from too little sharing and wants more isolation. Nothing here touches it.
  • Auditing every other unscoped Chroma read. This class likely exists elsewhere and deserves a sweep, but a speculative sweep is not this fix.
  • The workers:4 flip#15861 owns it and stays blocked on #15874.

Avoided Traps

  • Fixing only the failure that went red. The green one was the worse defect. Fixing only the observed failure would have left an assertion that cannot fail.
  • Treating it as isolation. It presented inside an isolation investigation, but it is wrong at any worker count.
  • Raising a limit or adding a retry. Both would have made red go green while leaving the assertion size-dependent.

Related

#15874 (parent investigation; cohort 2) · #15861 (blocked re-land) · #15878 / PR #15881 (the other non-isolation defect pulled out of the same matrix).

Handoff Retrieval Hints

  • query_raw_memories("QueryReRanker unscoped collection get unfalsifiable not.toContain truncation")
  • The polarity analysis is in #15874's cohort-2 trace comment.

Live sweep at 2026-07-24T23:33Z: no equivalent open ticket. A2A: no competing [lane-claim].