LearnNewsExamplesServices
Frontmatter
id15977
titleRestore scheduled corpus publication through the Data Sync Publisher identity
stateClosed
labels
bugairegressionarchitecturebuild
assigneesneo-gpt
createdAtJul 26, 2026, 9:21 AM
updatedAtJul 28, 2026, 7:30 PM
githubUrlhttps://github.com/neomjs/neo/issues/15977
authorneo-opus-ada
commentsCount10
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtJul 28, 2026, 6:07 PM

Restore scheduled corpus publication through the Data Sync Publisher identity

Closed Backlog/active-chunk-10 bugairegressionarchitecturebuild
neo-opus-ada
neo-opus-ada commented on Jul 26, 2026, 9:21 AM

Context

This ticket originally concluded that the corpus generator had never had an automated publisher. Live history falsifies that premise:

  • automated chore: ticket sync [skip ci] commits advanced resources/content/** until 2026-07-17;
  • the default-branch ruleset then began rejecting the local orchestrator's pushes with GH013;
  • the existing Data Sync workflow now has a working Publisher App identity and continues to advance dev, but its generated-path allowlist excludes resources/content/**;
  • the canonical long-running checkout therefore keeps regenerating the corpus locally without a delivery path. On 2026-07-28 it held 794 modified or untracked generated outputs.

This is a publication regression, not a missing-design investigation.

The Problem

The corpus currently has two incomplete halves:

  1. githubWorkflowSync still runs every two hours in a local maintainer checkout. It can emit issues, pull requests, discussions, and release notes, but its checkout credential cannot bypass the current dev ruleset.
  2. .github/workflows/data-sync-pipeline.yml already possesses the sanctioned Publisher App credential and successfully pushes generated data, but buildScripts/dataSyncPipeline.mjs neither runs the corpus generator nor stages resources/content/**.

Moving the existing generator into CI exposes two recovery edges:

  • the first discussion bootstrap has no updatedAt high-water mark and its first: 50 GraphQL query exceeds GitHub's resource limit; a measured outer page size of 30 succeeds while retaining complete cursor pagination;
  • GH_SyncService deliberately preserves successful facet output when another facet fails, but an ephemeral runner currently exits before that progress can be published.

The Fix

Extend the existing Data Sync pipeline as the sole scheduled corpus publisher:

  • run the canonical corpus generator in an emission-only mode under the workflow's read identity;
  • allowlist resources/content/** and publish it through the pipeline's existing Publisher App push;
  • cap the outer discussion bootstrap page at the measured safe size while retaining full pagination;
  • if one corpus facet fails, publish only the integrity-checked progress from successful facets, then fail the workflow loudly;
  • disable local scheduled corpus generation by default while retaining the manual syncGithubWorkflow entry point.

No second publisher identity or new ADR is needed: this extends the already-sanctioned Data Sync ownership and credential split.

Contract Ledger Matrix

Target Surface Source of Authority Required Behavior Failure Behavior Evidence
resources/content/** on dev GH_SyncService generators advances from the scheduled Data Sync workflow failed facet rolls back; successful facets may publish; workflow remains red focused unit tests + scheduled-run commit
Corpus read authority Data Sync reader context issues, pull requests, discussions, and releases are readable no write credential is exposed to generation workflow permissions/config inspection
Corpus push authority existing Publisher App final allowlisted generated diff is pushed to dev push failure fails the workflow existing authenticated push path + scheduled run
Local orchestrator orchestrator.localOnly.githubWorkflowSyncEnabled off by default; manual CLI remains available no periodic dirtying of maintainer checkouts config/unit coverage

Acceptance Criteria

  • Data Sync runs the canonical corpus generator in emission-only mode; that mode cannot commit or push.
  • The workflow grants only the read scopes required for pull requests and discussions in addition to its existing read scopes.
  • resources/content/** is part of the generated-data allowlist and is delivered only by the existing Publisher App final push.
  • A no-high-water discussion bootstrap uses an outer page size no larger than 30 and still follows every outer cursor.
  • When one corpus facet fails after another succeeds, integrity-checked successful-facet output is staged and pushed before the workflow reports the original failure.
  • Local scheduled githubWorkflowSync is disabled by default; the guarded manual maintenance command remains available.
  • [L4-deferred — operator handoff needed] An actual post-merge scheduled run advances resources/content/** on dev and the existing per-facet watchdog reports all facets fresh.
  • [L4-deferred — operator handoff needed] The canonical checkout's generated backlog is discarded only after the published commit proves that no generated data was lost.

Decision Record impact

None. The implementation extends the Publisher App / reader identity split already established by #15744 and the existing Data Sync pipeline.

Out of Scope

  • Durable within-facet cursor checkpoints and interrupted-run resume (#16002).
  • Discussion comment/reply completeness (#16016).
  • Corpus schema or rendering-format changes.
  • Any non-generated file cleanup in the canonical checkout.

Avoided Traps

  • Do not give the generator a write credential or let emission invoke its legacy nested push path.
  • Do not treat a green Data Sync run as corpus freshness until resources/content/** is in its allowlist.
  • Do not hide a partial facet failure merely because successful-facet progress was published.
  • Do not reset the canonical dirty checkout before the CI-published commit exists.

Related

  • #15744 — Publisher App / read-identity authority split
  • #15972 — green-pipeline / stale-corpus alarm
  • #16001 / PR #16015 — discussion updatedAt delta high-water support
  • #16002 — durable within-facet resume
  • #16016 — discussion comment/reply completeness

Retrieval Hint: resources content scheduled corpus publication Data Sync Publisher App GH013 discussion bootstrap

tobiu referenced in commit d7a664b - "chore(data): land the ISSUE facet of the ticket corpus (#15963) (#15964) on Jul 26, 2026, 3:39 PM
tobiu unassigned from @neo-opus-ada on Jul 28, 2026, 11:24 AM
tobiu referenced in commit 3fc2658 - "fix(ai): restore scheduled corpus publication (#15977) (#16093) on Jul 28, 2026, 5:22 PM
tobiu closed this issue on Jul 28, 2026, 5:22 PM
tobiu referenced in commit 7c32e7b - "fix(github-workflow): adapt discussion pagination (#15977) (#16097)" on Jul 28, 2026, 6:07 PM
tobiu closed this issue on Jul 28, 2026, 6:07 PM