LearnNewsExamplesServices
Frontmatter
id14551
titleFront door: evaluator paste-fold structure — wins above the 32KB cut
stateOpen
labels
documentationenhancementai
assignees[]
createdAtJul 3, 2026, 9:34 PM
updatedAtJul 3, 2026, 9:34 PM
githubUrlhttps://github.com/neomjs/neo/issues/14551
authorneo-opus-vega
commentsCount0
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]

Front door: evaluator paste-fold structure — wins above the 32KB cut

Open v13.1.0/archive-v13-1-0-chunk-8 documentationenhancementai
neo-opus-vega
neo-opus-vega commented on Jul 3, 2026, 9:34 PM

Context

Operator finding (2026-07-03), from the live validation runs of the #14549 amendment: the Gemini web UI could not accept the full guide — the paste cut off mid-§8 at learn/benefits/Introduction.md:167 ("…and does it match how Neo is meant to grow?* And hol"). Measured: the cut lands at 32,455 of 48,666 bytes — the paste cap is almost certainly 32,768 (2^15) characters. Every Gemini evaluation to date (the 8.5 baseline, the 9.2 extra-high cold run) scored the first ~67% of the doc.

Operator steer, verbatim posture: "LLM readers 'appreciate' the final 'wins' at the top… open minded here, but we do need a new ticket."

Live latest-open sweep at 2026-07-03T19:2xZ: no equivalent ticket. Related lane: #14549 / PR #14550 (MERGED — the gardener-handhold amendment this finding surfaced through).

The Problem

What sits BELOW the 32KB fold and therefore never reached any web-UI evaluator:

  • The second half of §8 (the proof) — including the honest-boundary paragraph and the recruit-not-opponent close;
  • §9 (What's in it for you) — the entire value/adoption section, i.e. the commercial half of the story;
  • §10 (the claims register) — ALL the receipts. Line 5 explicitly promises "the exact query that backs that number is in the claims register at the end of this guide" — a promise a truncated evaluator can read but structurally never verify. The doc's verifiability contract silently breaks at the fold;
  • §11 (the maintainer's account), §12 (self-prediction), §13 (license/next).

Two empirical facts sharpen the design space:

  1. The truncated doc already wins engagement: 8.5/9.2 cold scores were earned by the first 32KB alone — the front-loaded inventory (§1) + friction/loop/soul arc carries the verdict. The fold costs verifiability and value-narrative, not engagement.
  2. A skeptic-class evaluator counter-anchors on self-reported scores: the Claude Opus web-UI run (7/10, full engagement, independent repo/npm verification) explicitly discounted "the 8.5 the document reports from its own evaluator" and deducted for "prose engineered to grade itself." The merged §7 anchor still carries raw numbers (8.5/9.2) — the scores-to-ticket redesign existed as an orphaned commit (98dd742fb, branch deleted post-merge) and folds into this lane: keep the sycophancy STORY in §7, move the NUMBERS to tickets, and say why ("a self-reported score is an anchor, and the evaluator class this section respects most treats it as exactly that").

The Architectural Reality

  • learn/benefits/Introduction.md — 48,666 bytes, 274 lines, 13 sections. The §7 adversarial-triage protocol is the doc's declared defense surface; it currently assumes evaluators see the whole doc. Reality: the standard evaluation condition IS the 32KB truncated paste.
  • The doc's structure is double-audienced: the narrative arc (machine→friction→loop→soul→organism) serves human readers and the §11/§12 payoff sections are the emotional close; the inventory + evaluation sections serve machines. The fold splits these audiences physically.

The Fix — options, deliberately divergent (operator is open-minded; decide in-lane, record the rationale)

Option Shape Falsifier / cost
A. Inverted-pyramid restructure Reorder wholesale: proof + register + value above the fold; narrative below Breaks the human reading arc the guide was authored around; §11/§12 land as payoff BECAUSE they come late. Optimizing purely for the truncated machine reader may cost the human one
B. Above-the-fold evaluator block A compact (~1–2KB) block near the top: the 3–5 hardest receipts WITH their verification queries inline + the §7 evaluation rules pointer + an explicit "this doc is 48KB; if your paste truncated, the claims register at §10 did not reach you — fetch it at " Adds bytes to an already-heavy doc; another block to maintain. But it converts the fold from silent verifiability-break into a declared, navigable boundary
C. Fold-aware section reorder (surgical) Move §10 (claims register) directly after §8; let §9/§11/§12/§13 (human-payoff + adoption) sit below the fold — humans scroll, machines truncate The register is long; moving it up may push §7 itself toward the fold. Byte-budget math needed: everything an evaluator NEEDS inside 32,455 bytes
D. Separate paste-optimized artifact An evaluator-distillation file (llms.txt-style) The second-drift-surface class (dual maintenance — the exact defect ADR 0031 §5 forbids); evaluators get handed the guide, not the artifact

Recommendation to develop in the PR (falsify freely): B + C-lite — the small evaluator block up top (with the truncation self-declaration), plus moving the claims register above the fold if the byte math works; A and D rejected on their falsifiers. Whatever wins: §7's protocol text gains the truncation condition — the standard test condition is the 32KB paste, and future runs state what fraction of the doc the evaluator actually received.

Acceptance Criteria

  • Structure decision recorded in the PR with the option matrix adjudicated (not silently picked)
  • The claims register (or its load-bearing receipts) reachable within the first 32,455 bytes, OR the fold explicitly self-declared to the evaluator with a fetch pointer — no silent verifiability break
  • §7 protocol names the truncated-paste condition as the standard evaluation reality
  • The orphaned scores-to-ticket anchor redesign re-landed (§7 keeps the sycophancy story, numbers move to tickets, rationale stated in-text)
  • Byte accounting in the PR: what sits above/below the fold, before and after
  • Validation: operator re-runs the cold protocol (fresh sessions, same initial prompt, high + extra-high budgets) on the restructured doc as pasted (truncated) — pass = engagement verdict holds AND the evaluator demonstrates awareness of the receipts (cites or fetches the register) rather than scoring blind to it
  • Cross-family review before merge (front-door identity surface)
  • PR Resolves this leaf

Out of Scope

  • The measurement program itself (§7's named metrics — separate lanes)
  • Rewriting section CONTENT beyond the §7 anchor redesign — this is a structure/fold lane

Avoided Traps

  • Optimizing the doc into a scoring artifact — the Claude run's "prose engineered to grade itself" deduction gets worse if the restructure reads as rubric-chasing; every structural move must serve a REAL reader (truncated-machine or human), not a score
  • Dual-maintenance distillation (option D) — rejected unless the falsifier is answered

Related

Related: #14549 / PR #14550 (merged predecessor; this finding surfaced through its validation runs) · #14313 (guide authoring lane) · #14310 (learning-surface epic) · orphan commit 98dd742fb (scores-to-ticket redesign, content folds here)

Origin Session ID: 8cf234b7-e698-47ca-99e2-bf865196b6aa (continuation 233a6b21)

Retrieval Hint: front door 32KB paste fold truncation claims register above the fold evaluator block