bootstrapWorktree: generate dist/parse5.mjs (+ minimal build artifacts) so worktree can run full unit suite

Context

ai/scripts/bootstrapWorktree.mjs currently scopes to: gitignored ai/mcp/server/<name>/config.mjs copies + gitignored .neo-ai-data/ data subdirs symlinks. It does NOT generate build artifacts in the worktree's dist/ directory.

Empirical anchor 2026-05-10: running CI=true npm run test-unit (full suite) in a Claude Code worktree fails with:

Error: Cannot find module '/path/to/.claude/worktrees/<name>/dist/parse5.mjs'
  imported from /path/to/.claude/worktrees/<name>/src/functional/util/HtmlTemplateProcessor.mjs

Operator-flagged hypothesis: "dist/parse5 probably get generated by build all. might be a ai/scripts/bootstrapWorktree.mjs item. mostly important for the left hemisphere, but could always be done (takes less than 30s on this machine)."

Verified:

• dist/parse5.mjs generated by npm run bundle-parse5 per buildScripts/README.md
• buildScripts/util/templateBuildProcessor.mjs:2 imports from '../../dist/parse5.mjs'
• Main checkout /Users/Shared/github/neomjs/neo/dist/ has the full set: parse5, esm/, development/, production/, highlight/
• Fresh worktree dist/ has only ai-knowledge-base.jsonl (bootstrapped per current scope)
• Workaround today: scope test runs to AI-only substrate (npm run test-unit -- test/playwright/unit/ai/) which avoids parse5-dependent specs

The Problem

Without dist artifacts in a fresh worktree:

1. Full unit-suite verification (CI=true npm run test-unit) fails immediately
2. Any spec importing HtmlTemplateProcessor (functional component substrate) fails
3. Workaround: narrower spec-pattern scoping — but this leaves the worktree without true CI-equivalent verification capability
4. Recurring friction across Claude Code worktree sessions; operator-flagged today after running into it for the second time this session

The Architectural Reality

bootstrapWorktree.mjs distinguishes (per its own JSDoc):

• Source code: must be REAL COPIES (symlinks cause Neo.setupClass namespace collisions when worktree-local module + main-checkout module both register)
• Data directories: SAFE TO SYMLINK (no ESM import chains; better-sqlite3 + Chroma open by path transparently)

dist/parse5.mjs is a special case — it's a BUNDLED THIRD-PARTY module (not a Neo class), but it IS ESM-imported by templateBuildProcessor.mjs. Per the rule's spirit (namespace-collision), parse5 itself doesn't trigger collisions (it doesn't extend Neo.core.Base or register namespaces), so symlinking IS technically safe. But generating dist artifacts in the worktree is cleaner — keeps worktree self-contained, removes semantic-edge-case risk.

Same applies to other dist artifacts (esm/, development/, production/, highlight/) — bundled outputs from various npm run build-* scripts, ESM-imported by tests + runtime.

The Fix

Extend ai/scripts/bootstrapWorktree.mjs with a dist-generation step:

// After config copy + data symlinks + gitignored-file symlinks:
console.log('[bootstrap] Generating dist artifacts...');
spawnSync('npm', ['run', 'bundle-parse5'], {cwd: workTreeRoot, stdio: 'inherit'});
// Optionally: trigger broader build-all if operator wants full dist parity with main checkout

Scope decision points:

1. Minimal scope: just bundle-parse5 (sufficient for unit-suite-runs that import HtmlTemplateProcessor)
2. Full scope: entire build-all run (covers all dist artifacts; takes ~few minutes; matches main-checkout parity exactly)
3. Opt-in via CLI flag: --build-dist flag, off by default (minimizes default bootstrap time)

Operator-framing favors option 1 (<30s) or option 3 (opt-in for full).

Acceptance Criteria

• AC1: bootstrapWorktree.mjs generates dist/parse5.mjs (at minimum) post-config-copy + post-data-symlinks
• AC2: Fresh worktree post-bootstrap can run CI=true npm run test-unit without "Cannot find module dist/parse5.mjs" error (verify empirically)
• AC3: Existing bootstrap behavior unchanged (config copy + data symlinks + gitignored-file symlinks all preserved)
• AC4 (scope decision-AC): Either default-on minimal-scope (just bundle-parse5) OR opt-in CLI flag for broader build-all — implementation-time decision
• AC5: JSDoc updated to document the new dist-generation step + scope decision rationale

Out of Scope

• Symlinking dist/ to main checkout — rejected; mixes data semantics with build artifacts; potential edge cases with namespace collisions for any ESM-imported dist module
• Full build-all by default — too aggressive; minimal scope is sufficient for unblocking unit-suite + keeps default bootstrap time bounded
• Generating per-app build outputs — too aggressive; only platform-bundled-modules (parse5) needed for test substrate

Avoided Traps

• Skip-as-low-priority — operator-flagged today after empirical re-occurrence; small scope; high recurring-friction-reduction ROI for every Claude Code worktree session
• Bundle into broader bootstrap refactor — minimal change; bounded scope; no need to expand
• Symlink instead of generate — symlink semantics for ESM-imported modules has namespace-collision risk per bootstrap's own JSDoc rule, even though parse5 specifically wouldn't trigger it. Generation is cleaner architecturally.

Related

• Operator hypothesis surface: 2026-05-10 session, "dist/parse5 probably get generated by build all. might be a ai/scripts/bootstrapWorktree.mjs item."
• Empirical recurrence: 2 cases this session of full-unit-suite invocations failing on dist/parse5.mjs missing
• bootstrap JSDoc source-vs-data symlink rule: ai/scripts/bootstrapWorktree.mjs head comment
• parse5 build script: buildScripts/README.md + npm run bundle-parse5

Origin Session ID: c2912891-b459-4a03-b2af-154d5e264df1

Retrieval Hint: query_raw_memories(query="bootstrapWorktree dist parse5 generation unit suite full-run worktree friction")