Context / Friction
GH_SyncService.runFullSync() — invoked by both ai:sync-github-workflow (CLI) and the MCP sync_all tool — emits resources/content/ (issues/PRs/discussions/release-notes) and stops. The sitemap + index JSONs are derived from that content (buildScripts/docs/index/*.mjs → index JSONs, buildScripts/docs/seo/generate.mjs → sitemap.xml / llms.txt), but nothing fires the derive. So the invariant "content and its derived index are coherent" silently breaks on every manual/MCP sync — observed live as a 12996-vs-13257 gap (content fresh, sitemap stale), which makes the sitemap-driven middleware SSG skip the newest tickets.
The data-sync-pipeline.yml only stays coherent because it happens to run both (DevIndex content + an inlined "Rebuild Indexes & SEO" step that duplicates the 7 node commands).
Grounded findings
- The 4 content index builders + 2 SEO generators are local-only pure derives (no GH token/API; read
resources/content via fast-glob) — verified. The label index builder is a separate remote GitHub/SDK derive and should stay explicit for release/CI callers rather than becoming part of the sync_all post-hook.
- They are already importable functions (
createTicketIndex, getSitemapXml, getLlmsTxt, …) — buildScripts/release/prepare.mjs lines 138-155 already calls them as a bundle (its "Category B"), alongside release-only version-bump work ("Category A").
- Cost: index builders ~2.6s, SEO generator ~27s (CPU-bound) — cheap;
generate.mjs is already shallow-clone-aware (hasReliableGitHistory() preserves existing lastmod to avoid hourly churn), so CI placement is fine.
The fix (option B — operator-confirmed)
- Extract
prepare.mjs Category B (lines 138-155) into one named function — e.g. rebuildContentIndexesAndSeo() — a thin wrapper over the 7 already-imported functions.
- Reuse it in three callers:
prepare.mjs (release) — call it after the version bump.
data-sync-pipeline.yml — replace the 7 inlined node commands with one call (kills the duplication; drop the dead GH token on that step).
- A post-
sync_all hook in runFullSync()'s orchestration — so every content sync (CLI and MCP) regenerates the derived artifacts.
- (Defense-in-depth, optional) make the middleware
build.mjs derive its route list from the content tree, so a stale sitemap can't drop content.
Design decision to settle
Placement of the post-sync hook: runFullSync is doc-scoped to agent-OS content emission, so the derive should not be buried inside it (concern-mixing). Prefer a thin orchestration wrapper / post-emit hook that both the CLI and the MCP sync_all handler flow through — covering the MCP path is a hard requirement, or an agent syncing via sync_all re-opens the gap.
Contract Ledger Matrix
| Target Surface |
Source of Authority |
Proposed Behavior |
Fallback |
Docs |
Evidence |
rebuildContentIndexesAndSeo() shared derive helper (new) |
This ticket + current buildScripts/release/prepare.mjs derive bundle |
Export one named async helper that runs the local content index builders and regenerates apps/portal/sitemap.xml + apps/portal/llms.txt with canonical base URL https://neomjs.com; release/CI callers may opt into the remote label index explicitly |
If any selected derive fails, reject/throw so the caller fails instead of publishing incoherent derived artifacts |
JSDoc on the helper naming generated artifacts, includeLabelIndex, and failure behavior |
Focused unit coverage for helper sequencing/outputs plus node --check |
buildScripts/release/prepare.mjs release path |
Current release script order: version/package work first, then index+SEO derive |
Replace the inline Category B derive block with a call to the shared helper after release-only version work, preserving release-only behavior by passing includeLabelIndex: true |
Helper failure preserves current fail-fast release behavior |
Release script import/call remains self-explanatory; helper JSDoc owns detail |
Unit or static assertion that prepare.mjs imports/calls the shared helper; no duplicate inline seven-command bundle remains |
.github/workflows/data-sync-pipeline.yml Rebuild Indexes & SEO step |
Current workflow step duplicates seven node commands after DevIndex update; labels.mjs uses GH_LabelService and therefore still needs GitHub credentials |
Replace the inline seven commands with one repo-local helper invocation/command using --include-labels; keep GITHUB_TOKEN for the label derive because source inspection proves it is needed |
If helper exits non-zero, the workflow fails before committing stale derived artifacts |
Workflow step name may stay; no additional docs required |
Static test/grep assertion that the workflow uses the shared command and no longer contains the duplicated seven derive commands |
CLI GitHub Workflow sync path (ai/scripts/maintenance/syncGithubWorkflow.mjs) |
CLI currently invokes GH_SyncService.runFullSync() via the canonical SDK route |
A successful CLI content sync must run the shared local-only content derive after content emission and before reporting success to the operator |
If content sync fails, do not run derive; if derive fails after content sync, surface a non-zero/failed outcome rather than silent degradation |
JSDoc/comment at the orchestration boundary if a wrapper is introduced |
Focused unit coverage around the shared orchestration wrapper; CLI path delegates through it |
MCP sync_all path (ai/mcp/server/github-workflow/toolService.mjs) |
sync_all currently delegates through the dev-branch guard to SyncService.runFullSync() |
A successful MCP sync_all leaves content and derived index/SEO artifacts coherent via the local-only helper default; it must not import or execute the remote label derive |
If derive fails, the MCP call rejects/fails rather than returning success with stale sitemap/index data |
OpenAPI description only if observable semantics are changed; otherwise helper/orchestration JSDoc |
Unit/integration assertion that the MCP delegate path reaches the shared post-sync derive wrapper |
Acceptance Criteria
- A single
rebuildContentIndexesAndSeo() (or equivalent) is the one source of truth for the index+SEO derive.
prepare.mjs, the data-sync pipeline, and the content-sync path all call it — no inline duplication remains.
- A content sync via the MCP
sync_all tool leaves the sitemap coherent with the emitted content (the 12996-vs-13257 gap cannot recur).
Evidence: L2 (unit coverage of the extracted bundle + a coherence assertion that a sync leaves sitemap routes == content) → L2 required (deterministic local derive).
Refs #12964 (the middleware automation umbrella this serves).
Authored by Claude Opus 4.8 (Claude Code), @neo-opus-grace (Grace).
Context / Friction
GH_SyncService.runFullSync()— invoked by bothai:sync-github-workflow(CLI) and the MCPsync_alltool — emitsresources/content/(issues/PRs/discussions/release-notes) and stops. The sitemap + index JSONs are derived from that content (buildScripts/docs/index/*.mjs→ index JSONs,buildScripts/docs/seo/generate.mjs→sitemap.xml/llms.txt), but nothing fires the derive. So the invariant "content and its derived index are coherent" silently breaks on every manual/MCP sync — observed live as a 12996-vs-13257 gap (content fresh, sitemap stale), which makes the sitemap-driven middleware SSG skip the newest tickets.The
data-sync-pipeline.ymlonly stays coherent because it happens to run both (DevIndex content + an inlined "Rebuild Indexes & SEO" step that duplicates the 7nodecommands).Grounded findings
resources/contentvia fast-glob) — verified. The label index builder is a separate remote GitHub/SDK derive and should stay explicit for release/CI callers rather than becoming part of thesync_allpost-hook.createTicketIndex,getSitemapXml,getLlmsTxt, …) —buildScripts/release/prepare.mjslines 138-155 already calls them as a bundle (its "Category B"), alongside release-only version-bump work ("Category A").generate.mjsis already shallow-clone-aware (hasReliableGitHistory()preserves existing lastmod to avoid hourly churn), so CI placement is fine.The fix (option B — operator-confirmed)
prepare.mjsCategory B (lines 138-155) into one named function — e.g.rebuildContentIndexesAndSeo()— a thin wrapper over the 7 already-imported functions.prepare.mjs(release) — call it after the version bump.data-sync-pipeline.yml— replace the 7 inlinednodecommands with one call (kills the duplication; drop the dead GH token on that step).sync_allhook inrunFullSync()'s orchestration — so every content sync (CLI and MCP) regenerates the derived artifacts.build.mjsderive its route list from the content tree, so a stale sitemap can't drop content.Design decision to settle
Placement of the post-sync hook:
runFullSyncis doc-scoped to agent-OS content emission, so the derive should not be buried inside it (concern-mixing). Prefer a thin orchestration wrapper / post-emit hook that both the CLI and the MCPsync_allhandler flow through — covering the MCP path is a hard requirement, or an agent syncing viasync_allre-opens the gap.Contract Ledger Matrix
rebuildContentIndexesAndSeo()shared derive helper (new)buildScripts/release/prepare.mjsderive bundleapps/portal/sitemap.xml+apps/portal/llms.txtwith canonical base URLhttps://neomjs.com; release/CI callers may opt into the remote label index explicitlyincludeLabelIndex, and failure behaviornode --checkbuildScripts/release/prepare.mjsrelease pathincludeLabelIndex: trueprepare.mjsimports/calls the shared helper; no duplicate inline seven-command bundle remains.github/workflows/data-sync-pipeline.ymlRebuild Indexes & SEOstepnodecommands after DevIndex update;labels.mjsusesGH_LabelServiceand therefore still needs GitHub credentials--include-labels; keepGITHUB_TOKENfor the label derive because source inspection proves it is neededai/scripts/maintenance/syncGithubWorkflow.mjs)GH_SyncService.runFullSync()via the canonical SDK routesync_allpath (ai/mcp/server/github-workflow/toolService.mjs)sync_allcurrently delegates through the dev-branch guard toSyncService.runFullSync()sync_allleaves content and derived index/SEO artifacts coherent via the local-only helper default; it must not import or execute the remote label deriveAcceptance Criteria
rebuildContentIndexesAndSeo()(or equivalent) is the one source of truth for the index+SEO derive.prepare.mjs, the data-sync pipeline, and the content-sync path all call it — no inline duplication remains.sync_alltool leaves the sitemap coherent with the emitted content (the 12996-vs-13257 gap cannot recur).Evidence: L2 (unit coverage of the extracted bundle + a coherence assertion that a sync leaves sitemap routes == content) → L2 required (deterministic local derive).
Refs #12964 (the middleware automation umbrella this serves).
Authored by Claude Opus 4.8 (Claude Code), @neo-opus-grace (Grace).