LearnNewsExamplesServices
Frontmatter
titlefeat: Unified Data Sync Pipeline (#9869)
authortobiu
stateMerged
createdAtApr 10, 2026, 7:32 PM
updatedAtApr 10, 2026, 7:35 PM
closedAtApr 10, 2026, 7:35 PM
mergedAtApr 10, 2026, 7:35 PM
branchesdevagent/9869-data-sync-pipeline
urlhttps://github.com/neomjs/neo/pull/9870
Merged
tobiu
tobiu commented on Apr 10, 2026, 7:32 PM

Resolves #9869

Summary

The DevIndex automation pipeline originally only scraped opt-in/opt-out statuses and generated the users.jsonl output. This PR renames and hardens the pipeline into a unified Data Sync Pipeline that handles:

  1. Rebuild Indexes & SEO: Automatically runs index generation (labels.mjs, release.mjs, tickets.mjs) and SEO file generation (sitemap.xml, llms.txt).
  2. Content Synchronization: Copies the actual raw markdown ticket contents (issues, issue-archive) to the GitHub Pages root so the Portal Ticket Explorer does not hit a 404.
  3. Cross-Repo Atomicity: Propagates all updates back to neomjs/neo and neomjs/pages natively via git add -A logic safely executing cp.

Deviations

We integrated rm -rf plus git add -A instead of standard cp -r in the pages sync phase to natively mirror deleted files (i.e. tickets migrating to the archive).

tobiu
tobiu commented on Apr 10, 2026, 7:33 PM

Input from Antigravity (Gemini 3.1 Pro):

✦ # PR Review Summary

Status: Approved

Clinical evaluation of PR #9870: "feat: Unified Data Sync Pipeline (#9869)". The pipeline execution parameters precisely match the architectural intent established in the Swarm guidelines for data portability.


📊 Evaluation Metrics

  • [ARCH_ALIGNMENT]: 100 - Strict adherence to the neomjs/pages synchronization strategy utilizing atomic Git staging and GitHub Actions natively.
  • [CONTENT_COMPLETENESS]: 100 - Fat Ticket standard met (#9869). Deviation tracking and pipeline intents fully documented within the PR payload.
  • [EXECUTION_QUALITY]: 90 - Clean bash sequences leveraging rm -rf and git add -A to prevent "Ghost Files" from issue archival. Pipeline paths verified.
  • [PRODUCTIVITY]: 100 - All primary goals of #9869 achieved. SEO and Portal ticket data will stay synchronized hourly.
  • [IMPACT]: 70 - High operational impact. Automates major manual pain points associated with release index syncing and SEO generation.
  • [COMPLEXITY]: 35 - Low cognitive load but precise git choreography required.
  • [EFFORT_PROFILE]: Maintenance - Foundational operational maintenance shifting manual CI/CD dependencies to automation.

🕸️ Context & Graph Linking

  • Target Epic / Issue ID: Resolves #9869
  • Related Graph Nodes: Pipeline, SEO, GitHub Pages, Portal Ticket Explorer

🧠 Graph Ingestion Notes

  • [KB_GAP]: N/A
  • [TOOLING_GAP]: Originally, cp -r was considered, which fails to propagate tracked deletions. Addressed immediately via rm -rf targeting of the cloned repo prior to cp and strict reliance on git add -A.
  • [RETROSPECTIVE]: The "Fat Ticket" protocol has been successfully implemented and preserved within the GitHub Pull Request architecture. This ensures cross-swarm topological graphing remains structurally intact upon squash-merge.

📋 Required Actions

To proceed with merging, please address the following:

  • (Agent-Action) Completed. No further action needed. PR is ready for Squash-Merge.

The agent's task is considered "Done". Do not merge yourself.