LearnNewsExamplesServices
Frontmatter
id17240
title26.5 MiB of DevIndex data ships in every `npm i neo.mjs`
stateClosed
labels
bugaiperformancebuild
assigneesneo-opus-ada
createdAtAug 16, 2026, 9:16 PM
updatedAtAug 17, 2026, 9:22 AM
githubUrlhttps://github.com/neomjs/neo/issues/17240
authorneo-opus-grace
commentsCount0
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtAug 17, 2026, 9:22 AM

26.5 MiB of DevIndex data ships in every npm i neo.mjs

Closed Backlog/active-chunk-16 bugaiperformancebuild
neo-opus-grace
neo-opus-grace commented on Aug 16, 2026, 9:16 PM

Context

Measured 2026-08-16 with npm pack --dry-run --json against neo.mjs@13.1.0. The published package carries the DevIndex contributor corpus:

file size in tarball
apps/devindex/resources/data/users.jsonl 22.97 MiB
apps/devindex/resources/data/tracker.json 2.03 MiB
apps/devindex/resources/data/visited.json 1.50 MiB
total DevIndex data 26.5 MiB

Package totals: 39.1 MiB tarball, 96.2 MiB unpacked, 7,405 files. users.jsonl is the single largest file in the package — larger than the next four combined.

The Problem

Every npm i neo.mjs downloads and unpacks 26.5 MiB of GitHub contributor rankings. Nothing in the engine reads them. The only consumer is the DevIndex app's own store, which fetches the file over HTTP at runtime from the deployed site — apps/devindex/store/Contributors.mjs:50 resolves it from Neo.config.basePath, not from the package.

So the data is shipped to every engine consumer and read by none of them. It is roughly 27% of the unpacked package for a payload with zero engine consumers.

The cause is a stale exclusion rather than a missing one. .npmignore:4 reads:

apps/devindex/resources/*.json

Two independent reasons that no longer matches:

  1. Wrong directory. The corpus moved to apps/devindex/resources/data/. The pattern has no data/ segment and does not recurse.
  2. Wrong extension. The largest file is .jsonl, which *.json does not match even at the old path.

The rule was presumably correct when written. It silently stopped covering anything, and nothing observes an .npmignore entry that has quietly become vacuous.

The Architectural Reality

package.json declares no files array, so .npmignore is the sole gate on package contents. That makes every entry in it load-bearing and unverified — this is one confirmed miss, and the pattern class (a path-specific rule outliving the path) applies to the rest of the file.

The exclusions are also worth reading as a set rather than patching one line: the same npm pack output shows apps/portal/sitemap.xml (2.13 MiB) and apps/portal/llms.txt (1.09 MiB) shipping as well — generated SEO artifacts with no consumer inside an installed package. They are smaller and are not what this ticket is named for, but they are the same question asked of an adjacent path, and fixing one line while leaving them is the shape that produced this defect.

The Fix

Correct the exclusion so the DevIndex corpus cannot ship, in a way that does not depend on the data staying where it is today:

  • Exclude the DevIndex generated-data directory recursively and extension-independently, rather than restating a path+extension pair that can drift again.
  • Re-read the neighbouring generated-portal artifacts in the same pass and make an explicit keep/drop decision for each, recorded in the PR body.
  • Verify with npm pack --dry-run, which is the actual authority here — reasoning about .npmignore semantics is what produced the original miss.

Contract Ledger Matrix

Target Surface Source of Authority Proposed Behavior Fallback Docs Evidence
published neo.mjs tarball contents .npmignore (no files array in package.json) DevIndex generated data excluded none — exclusion is total .npmignore comments npm pack --dry-run --json before/after
apps/devindex/store/Contributors.mjs:50 runtime HTTP fetch from basePath unchanged — never read from the package n/a verified: the only consumer, and it is not package-resolved

Acceptance Criteria

  • npm pack --dry-run shows zero files under apps/devindex/resources/data/ in the tarball.
  • Tarball and unpacked sizes recorded before and after in the PR body, from the tool rather than from arithmetic.
  • The exclusion is written so that a future move or extension change within DevIndex's data directory stays covered — demonstrated by the pattern's shape, not asserted.
  • The DevIndex app still loads its data from the deployed site (its runtime path is HTTP, so this must be confirmed unaffected, not assumed).
  • Explicit recorded decision on apps/portal/sitemap.xml and apps/portal/llms.txt — excluded, or kept with a stated reason.

Out of Scope

  • The git-history side of the same corpus (#17238). This ticket changes only what ships to npm; it does not touch how the file is stored or synced.
  • Moving DevIndex out of this repository.
  • A files allowlist migration in package.json — a strictly better long-term shape than .npmignore, and a large enough behavior change to deserve its own ticket rather than riding this fix.

Avoided Traps

  • Patching the single line. Restating apps/devindex/resources/data/*.jsonl reproduces the exact failure mode — a rule pinned to today's path and extension.
  • Trusting the pattern over the tool. The original rule looked correct; npm pack is what shows the truth.

Related

#17238 (the same corpus in git history — different substrate, different fix) · apps/devindex/store/Contributors.mjs:50 (the sole consumer, HTTP-resolved)

Live latest-open sweep: latest 20 open issues checked 2026-08-16T19:14:37Z; no equivalent. A2A in-flight claim sweep: 30 most recent messages, newest 2026-08-16T14:10Z; no claim on this scope.

Origin Session ID: b17338dd-b474-494f-b08c-683044de2ddb Retrieval Hint: "npmignore stale path devindex users.jsonl ships in npm tarball npm pack dry-run"

tobiu referenced in commit e967fe0 - "feat(build): the npm package stops shipping the DevIndex corpus and Agent OS plane state (#17240) (#17253) on Aug 17, 2026, 9:22 AM
tobiu closed this issue on Aug 17, 2026, 9:22 AM