LearnNewsExamplesServices
Frontmatter
id12557
titleMake skill-manifest diff checks own changed semantics
stateClosed
labels
enhancementaibuildmodel-experience
assigneesneo-gpt
createdAtJun 5, 2026, 4:39 AM
updatedAtJun 5, 2026, 5:06 AM
githubUrlhttps://github.com/neomjs/neo/issues/12557
authorneo-gpt
commentsCount0
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
blockedBy[]
blocking[]
closedAtJun 5, 2026, 5:06 AM

Make skill-manifest diff checks own changed semantics

Closed v13.0.0/archive-v13-0-0-chunk-16 enhancementaibuildmodel-experience
neo-gpt
neo-gpt commented on Jun 5, 2026, 4:39 AM

Context

PR #12556 exposed a merge-gate friction in the skill-manifest reference-integrity lint that was added by the #12497 lane. The PR performs path-only doc-link moves for #12513, but CI fails Skill Manifest Lint because lint-skill-manifest.mjs --base origin/dev scans whole touched skill files and requires downstream docs sync whenever a skill changes, regardless of whether the changed lines actually alter the skill semantics.

Live latest-open sweep: checked the latest 20 open issues via GitHub Workflow MCP on 2026-06-05; #12513 is the adjacent blocked PR/ticket, but no equivalent open ticket covers this lint-semantics correction. Exact local sweep over resources/content/issues and resources/content/discussions for lint-skill-manifest, downstreamDocsTargets, section-ref, reference-integrity, changed-line, and #12497 found historical substrate and adjacent issues, but no live duplicate for this specific own-what-you-change failure. Semantic KB sweep was attempted and is currently unavailable (Knowledge Base collection not found), so this ticket records the fallback evidence rather than claiming semantic-KB coverage.

The Problem

The current --base behavior over-attributes pre-existing skill-substrate debt to mechanical PRs that touch a file for unrelated path updates:

  1. Section-reference integrity scans whole touched files. If a PR changes one link in .agents/skills/pr-review/references/pr-review-guide.md, the lint also reports unrelated pre-existing dangling § references elsewhere in the same file. PR #12556 reproduced this with stale refs already present on origin/dev.
  2. downstreamDocsTargets fires on path-only diffs. PR #12556 changed pull-request references from learn/agentos/evidence-ladder.md to learn/agentos/process/evidence-ladder.md. The downstream docs do not mention that moved target, yet the lint requires ProgressiveDisclosureSkills.md and CodebaseOverview.md to change solely because the skill changed.

This makes the #12497 guard too coarse: it catches true reference drift, but it also blocks correct mechanical refactors by forcing unrelated stale-debt cleanup or hollow downstream-doc edits into the same PR.

The Architectural Reality

Relevant substrate:

  • ai/scripts/lint/lint-skill-manifest.mjs owns the --base diff-aware skill-manifest checks.
  • .github/workflows/skill-manifest-lint.yml runs node ai/scripts/lint/lint-skill-manifest.mjs --base origin/dev on skill changes.
  • .agents/skills/skills.manifest.json owns downstreamDocsTargets metadata.
  • The #12556 failure log shows Skill Manifest Lint running on merge commit 2f7dcb41..., failing with stale section refs plus downstream docs targets while local non-base lint passes.
  • The intended discipline is still correct: changed skill semantics must not introduce broken references or leave downstream documentation stale. The defect is the ownership boundary, not the existence of the lint.

The Fix

Refine the diff-aware checks so --base mode enforces own what you change:

  1. Section-ref integrity should evaluate section refs introduced or modified by changed lines, not every section ref in every touched file.
  2. downstreamDocsTargets should skip link-path-only diffs when the downstream target docs do not mention the moved target and the skill description/contract semantics are unchanged.
  3. Keep current whole-file or full-manifest checks for non---base mode and for true skill contract changes.
  4. Add focused unit coverage for the PR #12556 failure shape: pre-existing stale section refs in a touched file are ignored when untouched, while a newly introduced stale ref is still rejected.
  5. Add focused unit coverage for downstream docs: semantic skill description changes still require downstream docs updates; path-only moved-doc references do not if downstream docs have no matching stale target.

Contract Ledger

Target Surface Source of Authority Proposed Behavior Fallback / Edge Case Docs Evidence
lint-skill-manifest.mjs --base section-ref check This ticket; #12497 reference-integrity intent; #12556 CI failure In --base mode, fail only section refs introduced or modified by the PR diff lines Non---base lint may remain whole-file; deleted/renamed files must not mask new broken refs Inline JSDoc / comments if helper shape is non-obvious Unit test with pre-existing stale refs + newly introduced stale ref
downstreamDocsTargets enforcement skills.manifest.json downstream-doc contract; #11275 original manifest discipline Require downstream docs updates when skill semantics/description change; skip pure link-path-only moves when downstream docs do not cite the moved target If path-only diff changes a target that downstream docs do cite, lint should either require downstream sync or verify no stale downstream refs remain Lint error wording should explain the skip/require discriminator Unit test for semantic change vs path-only moved-doc reference
skill-manifest-lint.yml CI behavior Current CI workflow Existing workflow keeps invoking the same script with --base origin/dev; semantics improve in the script, not by weakening CI CI still fails on true new broken refs or real downstream-doc drift No workflow docs required unless command changes node ai/scripts/lint/lint-skill-manifest.mjs --base origin/dev green on #12556-style diff

Decision Record Impact

none. This is a lint-behavior refinement aligned with the existing skill-manifest governance substrate.

Acceptance Criteria

  • lint-skill-manifest.mjs --base <ref> scopes section-ref integrity to changed lines or an equivalent ownership-preserving diff unit.
  • A pre-existing stale section ref elsewhere in a touched file does not fail a PR when the PR does not modify that line.
  • A newly introduced or modified stale section ref still fails.
  • downstreamDocsTargets does not require downstream-doc changes for path-only moved-doc reference updates when the downstream docs do not cite the moved target and skill semantics are unchanged.
  • Semantic skill description/contract changes still require downstream-doc sync.
  • Focused unit coverage covers both discriminators.
  • node ai/scripts/lint/lint-skill-manifest.mjs --base origin/dev passes on the #12556-style path-only relocation case without weakening true-positive failures.

Out of Scope

  • Fixing all pre-existing dangling § references across skill substrate.
  • Weakening NO_TOP_LEVEL_ORPHAN or reshaping #12556's learn/agentos/process/ relocation.
  • Removing downstreamDocsTargets; this ticket refines when it fires.
  • Broad replacement of the skill-manifest governance model.

Avoided Traps

  • Allowlist heavily referenced process docs at top-level. Rejected. Ref-density is not the discriminator; structural path expectation is. AGENTS_ATLAS stays top-level because turn-loaded AGENTS.md expects that path, while methodology docs can move to process/ with updated references.
  • Absorb stale section-ref cleanup into #12556. Rejected as the preferred path because it mixes unrelated debt into an orphan-cleanup PR. It remains an emergency unblock option, not the right lint substrate fix.
  • Disable the CI guard. Rejected. The guard catches real drift; the fix is narrower ownership semantics.

Related

  • #12497 — reference-integrity lint lane that surfaced this coupling.
  • #12513 — learn/agentos orphan cleanup ticket blocked by this over-attribution.
  • PR #12556 — reproduces the failure with process-doc relocation.
  • #11275 — skill-manifest governance substrate and downstreamDocsTargets lineage.

Origin Session ID: dcdaac0b-9ae0-45b5-b4da-da39541af497

Retrieval Hint: "lint-skill-manifest changed lines downstreamDocsTargets path-only diff section-ref pre-existing debt #12556 #12497"

tobiu referenced in commit 099e30e - "fix(ai): scope skill manifest diff ownership (#12557) (#12558)" on Jun 5, 2026, 5:06 AM
tobiu closed this issue on Jun 5, 2026, 5:06 AM