LearnNewsExamplesServices
Frontmatter
id12999
titlePortal news deep links: O(all-chunks) sequential resolution stalls archive routes
stateClosed
labels
bugaiperformance
assigneesneo-fable-clio
createdAtJun 12, 2026, 3:47 PM
updatedAtJun 12, 2026, 9:37 PM
githubUrlhttps://github.com/neomjs/neo/issues/12999
authorneo-fable-clio
commentsCount0
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
blockedBy[]
blocking[]
closedAtJun 12, 2026, 9:37 PM

Portal news deep links: O(all-chunks) sequential resolution stalls archive routes

Closed v13.1.0/archive-v13-1-0-chunk-1 bugaiperformance
neo-fable-clio
neo-fable-clio commented on Jun 12, 2026, 3:47 PM

Context

Found while unblocking the middleware SSG remainder build (the 4,138 unbuilt ticket routes, #12964 thread): batch after batch of archive ticket routes died at the renderer's 120s kill while active tickets rendered in ~3.6s. Three rounds of instrumentation (fetch miss-path tracer → settle-loop tracer → fetch+boolean tracer) proved the SSR environment innocent: every fetch is served, initVnode resolves every attempt — the app simply never requests the ticket's markdown, because the deep-linked record never loads.

Release classification: ON the operator's active release thread — the v13 web-presence work (GitHub Pages update + middleware SSG) blocks on it; operator-directed pickup ("please go ahead").

The Problem

ensureRecordLoaded(itemId) in the three news controllers (tickets :98, pulls :109, discussions :98 under apps/portal/view/news/*/MainContainerController.mjs) resolves a deep-linked id by loading every unloaded folder chunk sequentially until the id happens to appear:

folders = store.items.filter(record => !record.isLeaf && record.childrenUrl && !record.isChildrenLoaded);
for (i = 0, len = folders.length; i < len; i++) {
    await tree.onFolderItemClick(folders[i]);   // one awaited chunk fetch per iteration
    record = store.get(itemId);
    if (record) { ... return record }
}

The tickets tree has 1,045 chunks. A v8.x archive ticket sits hundreds of folders deep in tree order → hundreds of sequential fetch+insert cycles → minutes per deep link, on the live site exactly as in SSR. The SSG hits all ~8,500 archive ticket routes, so the build burns the 120s timeout per route (~17h of mostly-nothing — the historical "8h local" pain was this). Live-site users deep-linking any archived ticket (e.g. from old external links or search results once the SSG lands) hit the same stall.

The Architectural Reality

  • The chunk loaders: src/app/content/TreeList.mjs#onFolderItemClick (:158, lazy, per-folder fetch via getLazyChildUrl = lazyChildUrlPrefix + childrenUrl); prefix '../../apps/portal/resources/data/' in all three MainContainers (:51).
  • The id→chunk knowledge already exists at build time: the three index writers (buildScripts/docs/index/{tickets,pulls,discussions}.mjs) iterate every item with its chunk assignment when building the chunked surface (root index + leaf files + manifest).
  • The store/tree layer needs the folder record id to load exactly the right chunk: store.get(chunkFolderId)tree.onFolderItemClick(folder) → leaf records insert.
  • SSR settle couples to this: isMarkdownLoaded stays false until the record's content path resolves → the 360-attempt loop runs to exhaustion under build concurrency (middleware SsrService settle loop).

The Fix

  1. Writers emit an id→chunk map alongside each chunked surface: <type>/idMap.json = flat {"<itemId>": "<chunkFolderId>"} (values = the root-index folder ids the tree already holds). Derivable inside the existing per-item loop — no new scan. (~9.2k entries for tickets ≈ ~300KB raw / ~70KB gzipped, lazy-fetched only on a deep-link miss; a compacted chunks-array+index encoding is noted as future-proofing if size ever matters.)
  2. Controllers consult the map first: on a store miss, fetch lazyChildUrlPrefix + '<type>/idMap.json' (memoized per controller), store.get(map[itemId]) → load exactly that folder via tree.onFolderItemClick → return the record. The existing sequential cascade stays as the fallback for ids absent from the map (fresh items between syncs) — graceful, never worse than today.
  3. The TicketIndex spec gains idMap assertions; pulls/discussions writer specs extended if present.

Contract Ledger Matrix

Target Surface Source of Authority Proposed Behavior Fallback Docs Evidence
apps/portal/resources/data/{tickets,pulls,discussions}/idMap.json (NEW ×3) the three index writers Flat {itemId: chunkFolderId}; regenerated with each index emit Absent file → controllers use the legacy cascade (404 tolerated) Writer JSDoc Writers' existing per-item chunk assignment loops (tickets.mjs buildChunkedIndex, siblings mirror)
ensureRecordLoaded (EXISTING ×3, modified) apps/portal/view/news/*/MainContainerController.mjs Map-first single-chunk resolution; cascade demoted to fallback Map miss / fetch failure → cascade unchanged Method JSDoc Controller sources read in full; cascade verified as the stall via SSR boolean tracer
src/app/content/TreeList.mjs (EXISTING — consumed, not modified) :111-124,158-200 getLazyChildUrl prefix convention reused for the map URL; onFolderItemClick invoked per resolved folder unchanged unchanged Loader read in full
Middleware SSG settle (EXISTING — consumer, no change in this PR) middleware SsrService Ticket routes settle in seconds once the record loads deterministically n/a n/a Probe evidence: active ticket 3.6s (record found early) vs archive timeout (record never found)

Decision Record impact: none.

Acceptance Criteria

  • Each writer emits <type>/idMap.json mapping every leaf id to its chunk folder id; regenerating against the repo produces a map whose entry count equals the emitted leaf count.
  • ensureRecordLoaded resolves a deep-linked ARCHIVE id with exactly ONE chunk fetch after the map fetch (verified by the SSR fetch trace shape or spec-level fetch counting).
  • Map-miss path: an id absent from the map still resolves via the legacy cascade (no regression for fresh items).
  • TicketIndex.spec asserts the idMap content for its fixture tickets (active + archived); 3/3+ green.
  • SSR probe evidence: a previously-timing-out archive ticket route (e.g. /news/tickets/6193) renders to completion in single-digit seconds with the markdown present.
  • Pulls + discussions controllers receive the identical map-first treatment (one pattern, three instances — the #12895 one-ticket precedent).

Out of Scope

  • Middleware-side changes (the SSG build consumes this via dep refresh; the settle-loop hardening stays #12964 Phase-1).
  • Compacted map encodings (noted as future-proofing).
  • Whitebox-e2e deep-link coverage (follow-up candidate once the SSG run validates at scale).
  • The eager all-backlog-chunks load on plain route entry (separate smaller inefficiency, observed in the trace; not the stall).

Avoided Traps

  • Per-chunk id ranges in the manifest — id contiguity within chunks is not a guaranteed invariant of the sync pipeline; an explicit map is truthful, ranges would be an assumption.
  • SSR-side injection of the chunk hint — fixes only the build, leaves live-site deep links stalling; the defect is app-layer.
  • Replacing the cascade outright — fresh-item ids can exist in content before the next index emit; the fallback preserves liveness.

Related

#12964 (the middleware automation thread this unblocks), #12981/PR #12998 (the monolith deletion this work surfaced under), #12957 (the settle/wedge sibling), #12895 (the one-pattern×3-siblings ticket precedent).

Live latest-open sweep: checked latest 12 open issues at 2026-06-12T13:40Z; no equivalent found. A2A in-flight sweep: clean (recent claims: #12994 gpt, #12995 vega, #12996 fable — disjoint scopes).

Origin Session ID: 3e1566f6-6336-4db4-8726-189004578d8b

Retrieval Hint: "ensureRecordLoaded idMap chunk deep link archive ticket SSG stall sequential cascade"

tobiu referenced in commit f1850f3 - "feat(portal): deterministic news deep links via build-emitted id maps (#12999) (#13003) on Jun 12, 2026, 9:37 PM
tobiu closed this issue on Jun 12, 2026, 9:37 PM