LearnNewsExamplesServices
Frontmatter
id13348
titleActive-tier re-chunk never persists — discarded by the data-sync
stateClosed
labels
bugaibuild
assigneesneo-opus-grace
createdAtJun 15, 2026, 3:37 PM
updatedAtJun 15, 2026, 3:46 PM
githubUrlhttps://github.com/neomjs/neo/issues/13348
authorneo-opus-grace
commentsCount1
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
blockedBy[]
blocking[]
closedAtJun 15, 2026, 3:46 PM

Active-tier re-chunk never persists — discarded by the data-sync

neo-opus-grace
neo-opus-grace commented on Jun 15, 2026, 3:37 PM

Context

Follow-up to #13328 / #13332. #13332 added reconcileActiveChunks (the active-tier ordinal-100 rebalance) to heal the portal's chunk drift, and the portal index numbers are now correct on dev. But while advancing the L4 portal-render verification residual @neo-gpt flagged on the #13332 review (PRR_kwDODSospM8AAAABDAicQw), a V-B-A on origin/dev showed the content is still drifted: the re-chunk's renames never persist.

The Problem

reconcileActiveChunks is wired only into buildScripts/docs/rebuildContentIndexesAndSeo.mjs — the index-rebuild step the hourly data-sync-pipeline.yml runs. That workflow's commit step stages only apps/portal/resources/data + SEO (sitemap.xml, llms.txt) and then git reset --hard (.github/workflows/data-sync-pipeline.yml:73) discards every other working-tree change — including the re-chunk's resources/content/** renames.

Net effect (V-B-A'd on origin/dev):

  • The index is regenerated from the re-chunked working tree and committed → portal nav numbers are correct (tickets active-chunk-1..4 = 100/100/100/15; pulls active-chunk-1..2).
  • The underlying content is reset → resources/content/issues stays ordinal-drifted (chunk distribution ~75/89/100/29/9/5/2/1/1/2/3, not 100/100/100/15).
  • → index ↔ content desync (deep-link / _index.json lookups resolve against the re-chunked layout the committed content no longer matches), and the re-chunk re-runs-and-discards every hour, never converging.

The data-sync pipeline is not the content-commit owner. resources/content/** is committed out-of-band by SyncService (the "chore: ticket sync [skip ci]" and "Content Sync" commits).

The Architectural Reality

  • ai/services/github-workflow/SyncService.mjs is the content-commit owner. It already runs the sibling reconciles before committing: IssueSyncer.reconcileClosedIssueLocations (:130) and PullRequestSyncer.reconcileClosedPullRequestLocations (:136) — the active→archive transition for closed items.
  • It does not run the active-tier ordinal rebalance. So after a delta sync drifts the active chunks, nothing in the content-commit path rebalances them; only the (discarded) index-rebuild does.
  • reconcileActiveChunks (ai/services/github-workflow/shared/reconcileActiveChunks.mjs) is idempotent (re-running on already-correct content is a no-op), so it is safe to call from more than one site.

The Fix

Call reconcileActiveChunks in SyncService, after the closed-location reconciles (:130/:136) and before the content-commit, for issues + pulls + discussions. The active-tier rebalance then lands in the "ticket sync" / "Content Sync" commit (the content-commit path), so the content converges to the ordinal-100 layout and stays in sync with the index.

Keep the existing rebuildContentIndexesAndSeo call as index-side defense (idempotent; keeps the index correct even if a data-sync runs before the next SyncService pass).

Decision Record impact

aligned-with ADR 0004 (universal ordinal-100 chunking) — enforces the ADR's active-tier invariant in the content-commit path, where it was missing.

Acceptance Criteria

  • SyncService calls reconcileActiveChunks for issues, pulls, and discussions after the closed-location reconciles and before the content-commit.
  • A unit test asserts the active re-chunk runs in the sync path (seam/spy on reconcileActiveChunks, asserted in the SyncService sync sequence) — mirroring how the existing closed-location reconciles are covered.
  • After a sync that drifts active chunks, the committed resources/content/{issues,pulls,discussions} converges to the ordinal-100 layout (each active chunk = 100 except the last).
  • The rebuildContentIndexesAndSeo call site is retained (index-side defense).

Out of Scope

  • The L4 portal-render verification residual itself (the #13332 review follow-up) — this ticket fixes the content-side cause; render verification stays its own task.
  • Prod deploy freshness / Cloud Run re-deploy automation (#12964) — separate axis.
  • Fixing the delta-syncers to not drift in the first place (the reconcile is the post-hoc heal by design).
  • Making the data-sync-pipeline.yml commit-staging stage resources/content (rejected: a second committer of resources/content collides with SyncService's ownership + the workflow's git reset --hard/rebase model).

Related

  • Parent: #13328 (portal chunk drift) → #13332 (the index-side re-chunk, merged).
  • Sibling follow-up: the #13332 review residual — L4 portal-render verification (PRR_kwDODSospM8AAAABDAicQw).
  • #12964 — middleware SSR rebuild + Cloud Run deploy automation (deploy axis).
  • ADR 0004 — universal ordinal-100 chunking.

Handoff Retrieval Hints

  • Retrieval Hint (semantic): "active-tier re-chunk discarded by data-sync reset; SyncService content-commit owner"
  • Retrieval Hint (commit anchors): #13332 head f1d723a20f0e95c8e1cf572e649f44872a2381d1; .github/workflows/data-sync-pipeline.yml:73 (the git reset --hard); SyncService.mjs:130/136 (the sibling reconciles).

Live latest-open sweep: checked latest 25 open issues at 2026-06-15 (~13:35Z) + A2A claim sweep (30 recent) + git grep origin/dev; no equivalent ticket or in-flight claim found. KB semantic sweep unavailable (collections error) — live + A2A + grep sweeps sufficient.

Release classification: boardless (v13 shipped; data-pipeline correctness, not release-gating).