LearnNewsExamplesServices
Frontmatter
id12582
titleValidate named section refs in skill-manifest lint
stateClosed
labels
enhancementaitestingbuildmodel-experience
assigneesneo-gpt
createdAtJun 5, 2026, 7:30 PM
updatedAtJun 5, 2026, 8:42 PM
githubUrlhttps://github.com/neomjs/neo/issues/12582
authorneo-gpt
commentsCount0
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
blockedBy[]
blocking[]
closedAtJun 5, 2026, 8:42 PM

Validate named section refs in skill-manifest lint

Closed v13.0.0/archive-v13-0-0-chunk-16 enhancementaitestingbuildmodel-experience
neo-gpt
neo-gpt commented on Jun 5, 2026, 7:30 PM

Context

PR #12581 surfaced a non-blocking tooling gap during cross-family review: converting numeric skill section references such as §6.2 into named section references can make the current skill-manifest lint stop validating the reference rather than proving the new anchor is valid.

This matters because the skill substrate now contains named section anchors and references such as audits/consensus-mandate.md §template-block, while the current lint path only resolves numeric §N.N section IDs.

Live latest-open sweep: checked the latest 20 open issues at 2026-06-05T17:29:00Z via gh issue list --state open --limit 20 --json number,title,author,labels,url; no equivalent open ticket found.

Duplicate/content sweep:

  • Exact GitHub search for named section refs lint-skill-manifest found only closed #11187, not equivalent.
  • Exact/semantic sweeps for sectionRefPattern named anchors, skill reference integrity lint named, and lint-skill-manifest section ref found historical related work but no open duplicate.
  • Historical related ticket #11560 is closed and covered semantic-anchor lint for agent refs in a different lint surface; it does not close this lint-skill-manifest named-section validation gap.

The Problem

checkSkillReferenceIntegrity() currently catches dangling numeric section references, broken relative markdown pointers, and manifest prose refs. It does not validate named section-reference forms. That creates a false-positive green path: a PR can replace a numeric section reference with a named section reference and pass lint even if the target named heading is stale or missing.

The failure mode is subtle because the source file pointer can still be valid. The file exists, but the named section after § is not resolved against the target file's headings.

The Architectural Reality

Verified current substrate:

  • ai/scripts/lint/lint-skill-manifest.mjs:605 documents checkSkillReferenceIntegrity() as checking resolvable markdown pointers and numeric section references.
  • ai/scripts/lint/lint-skill-manifest.mjs:638 defines sectionRefPattern with §(\d+\.\d+(?:\.\d+)*), so only numeric section IDs are captured.
  • ai/scripts/lint/lint-skill-manifest.mjs:654-658 resolves captured numeric IDs against the target file's headingIndex and reports dangling section ref when missing.
  • test/playwright/unit/ai/scripts/lint/lintSkillManifest.spec.mjs:475-543 covers numeric dangling refs and changed-line ownership behavior.
  • test/playwright/unit/ai/scripts/lint/lintSkillManifest.spec.mjs:574-589 covers manifest prose refs, again using numeric section IDs.
  • test/playwright/unit/ai/scripts/lint/lintSkillManifest.spec.mjs:592+ covers broken relative markdown pointers.
  • learn/agentos/decisions/0008-skill-anatomy-and-authoring-contract.md:125 names ai/scripts/lint/lint-skill-manifest.mjs as the skill manifest enforcement layer.

The Fix

Extend the skill reference-integrity lint so named section references are either validated or explicitly rejected by policy. Preferred shape: validate named anchors instead of banning them, because named section anchors already exist in skill audit payloads and improve semantic stability when headings move.

Implementation should stay in the existing lint surface:

  • Extend checkSkillReferenceIntegrity() / helper parsing in ai/scripts/lint/lint-skill-manifest.mjs to recognize named section references after §, not only numeric §N.N forms.
  • Normalize the named token against the existing heading index or an explicitly added named-anchor index.
  • Preserve existing numeric ref behavior and broken-file-pointer behavior.
  • Preserve changed-line ownership when --base supplies changedLinesByRelPath; do not make old untouched refs suddenly block unrelated PRs.
  • Add focused unit coverage in test/playwright/unit/ai/scripts/lint/lintSkillManifest.spec.mjs.

If implementation analysis proves named refs should be disallowed instead, the PR must document that policy choice and add a lint failure for named section references, rather than silently ignoring them.

Contract Ledger Matrix

Target Surface Source of Authority Proposed Behavior Fallback / Edge Case Docs Evidence
checkSkillReferenceIntegrity() in ai/scripts/lint/lint-skill-manifest.mjs Existing lint implementation at ai/scripts/lint/lint-skill-manifest.mjs:605-658; ADR 0008 lists this script as the skill-manifest enforcement layer Validate named section references in skill markdown/prose in addition to numeric §N.N refs Missing target file remains a broken pointer error; missing named heading becomes a dangling section-ref error; changed-line ownership remains respected under --base Update the @summary or nearby JSDoc if the accepted contract broadens beyond numeric refs Focused unit tests for valid named ref, dangling named ref, file target plus named ref, and changed-line scoped behavior
lintSkillManifest.spec.mjs reference-integrity tests Existing numeric/manifests/pointer tests at test/playwright/unit/ai/scripts/lint/lintSkillManifest.spec.mjs:475-610 Encode the named-ref contract so the lint cannot regress silently Existing numeric tests must remain green No public docs required unless implementation chooses a new named-ref syntax policy Targeted Playwright unit invocation for test/playwright/unit/ai/scripts/lint/lintSkillManifest.spec.mjs
Skill markdown references in .agents/skills/** ADR 0008 skill anatomy; existing skill audit files using named §... anchors and references Skill authors can rely on lint to catch stale named section references If named refs are rejected, lint must produce an actionable error that tells authors to use numeric refs or another supported form Only update skill-authoring guidance if policy changes author behavior npm run ai:lint-agents or equivalent lint command plus the focused unit suite

Decision Record impact

Aligned with ADR 0008. No ADR amendment is expected if this only extends enforcement coverage for the existing skill substrate. If the implementation changes the allowed skill-reference syntax policy, update or cite the relevant skill-authoring guidance as part of the PR.

Acceptance Criteria

  • lint-skill-manifest validates at least one documented named section-reference form used by .agents/skills/**.
  • A named section reference to a missing heading fails with an actionable dangling section ref-class error.
  • Relative file targets from nested skill payload directories still resolve before named-section validation.
  • Existing numeric §N.N section-ref behavior remains covered and green.
  • Existing broken markdown pointer behavior remains covered and green.
  • --base / changed-line ownership remains respected for named section refs.
  • Focused unit tests cover valid named refs, dangling named refs, file target plus named refs, and changed-line scoped behavior.
  • The PR records the validation commands used; do not use the default unscoped npx playwright test command.

Out of Scope

  • Repo-wide rewrite of all skill section references.
  • Renaming skill headings.
  • Adding a new skill or changing the progressive-disclosure loading model.
  • Changing PR #12581; that PR remains a separate identity-handle propagation lane.

Avoided Traps

  • Treating a green lint result after numeric-to-named conversion as proof that the reference was validated. Today that can mean the reference exited lint coverage.
  • Solving this by broad repo churn. The durable fix belongs in the existing lint parser and tests.
  • Making all historical named refs block unrelated PRs. Changed-line ownership exists to prevent that review-loop failure mode.

Related

Related: #12580, #12493, #12557, #11560. Follow-up surfaced during PR #12581 review.

Origin Session ID: dbb1a88c-987f-4519-9645-8f13e9d71000

Handoff Retrieval Hint: lint-skill-manifest named section references PR #12581 tooling gap

tobiu referenced in commit 9f16841 - "fix(lint): validate named skill section refs (#12582) (#12584)" on Jun 5, 2026, 8:42 PM
tobiu closed this issue on Jun 5, 2026, 8:42 PM