Add size-budget retention for MCP server logs

Context

We hardened the MCP server log path with two separate fixes:

• Neural Link bridge payload logging is now bounded by bridgePayloadDebugMaxChars from the Neural Link state provider (ai/services/neural-link/ConnectionService.mjs:17, ai/services/neural-link/ConnectionService.mjs:552, ai/mcp/server/neural-link/config.mjs:111).
• The shared MCP logger now prunes historical files by age/count before opening the active stream (ai/mcp/server/shared/logger.mjs:195, ai/mcp/server/shared/logger.mjs:240, ai/mcp/server/shared/logger.mjs:320). MC, KB, and NL expose loggerRetention.enabled, maxAgeDays, and maxFiles leaves (ai/mcp/server/memory-core/config.mjs:460, ai/mcp/server/knowledge-base/config.mjs:245, ai/mcp/server/neural-link/config.mjs:89).

That means the premise is not "logs grow forever". The residual gap is narrower: age/count retention can still leave a single historical spike on disk for the full age window.

Release classification: post-release/cloud-dogfooding hardening; useful for long-running harnesses, not a release blocker.

The Problem

Local dogfood evidence showed .neo-ai-data/logs at 152M, dominated by one historical file: .neo-ai-data/logs/nl-server-2026-06-15.log at 106M. Current retention would keep that file until the 14-day age policy expires because file count is still under the 30-file cap for that prefix.

That is acceptable as an immediate non-outage state, but it is not a bounded disk-budget contract for long-running MCP fleets. One bursty day can consume a disproportionate amount of the log directory even after payload-level logging is fixed.

Duplicate / prior-art sweep at 2026-06-19T04:07:09Z:

• Live latest-open sweep: checked latest 20 open issues; no equivalent MCP log byte-budget retention issue found.
• A2A in-flight claim sweep: checked latest 30 all-status messages; no competing [lane-claim] / [lane-intent] for MCP log byte budgets found.
• Semantic KB sweep: surfaced archived #10582 / #10576 daily-rotation tickets, where disk-bloat policy was explicitly out of scope; no current max-byte retention ticket surfaced.
• Exact sweep: rg "maxTotalBytes|maxBytes|maxSize|maximum bytes|size budget|byte budget" ai resources/content/issues resources/content/archive/issues resources/content/discussions found no MCP logger size-budget implementation or open issue.

The Architectural Reality

The shared logger is the right substrate. selectPrunableLogFiles() currently selects historical files exceeding maxAgeDays or maxFiles, and pruneLoggerRetention() excludes the active current-day file before selection. That active-stream guarantee should remain intact.

Config must remain state-provider-owned. ADR 0019 says AiConfig is the reactive Provider SSOT: consumers read resolved leaves at the use site, and config defaults/env bindings live in the provider leaf. This ticket must not introduce module-level default constants that diverge from the provider leaves.

The Fix

Add an optional size-budget dimension to the existing shared MCP logger retention policy.

Suggested shape:

• Add per-server state-provider leaves, e.g. loggerRetention.maxTotalBytes, with env bindings for MC / KB / NL.
• Extend resolveLoggerRetention() and selectPrunableLogFiles() so historical files matching a prefix are pruned oldest-first until the historical retained set fits the size budget.
• Preserve the active current-day file unconditionally.
• Treat missing, null, non-finite, or non-positive size values as "size dimension disabled" unless the implementation chooses a stricter validation policy and documents it.
• Keep prune failures warning-only; logging must not crash MCP server startup.
• Cover the selector and pruning behavior with focused unit tests using fake file metadata/sizes.
• Update config-template parity tests for any new env leaves.

Contract Ledger Matrix

Decision Record impact

Aligned with ADR 0019. The ticket adds provider-owned leaves and use-site reads; it must not add hardcoded retention defaults outside the state-provider config layer.

Acceptance Criteria

• MC, KB, and NL configs expose a size-budget retention leaf through the existing state-provider pattern.
• No new logger-retention default is re-declared as a module-level constant outside the provider leaf.
• The shared logger prunes oldest historical files for a prefix until the size budget is satisfied.
• The active current-day file is always preserved, even when the historical budget is exceeded.
• Existing age/count retention behavior remains covered and unchanged.
• Invalid or disabled size-budget config does not crash server startup.
• Focused unit tests cover size-budget selection, active-file preservation, prefix isolation, and prune failure behavior.
• Config-template parity tests cover the new env leaves.

Out of Scope

• Compressing logs.
• Manual cleanup of existing local log files.
• Changing Neural Link bridge payload logging again.
• Adding MCP healthcheck output for log disk usage.
• Changing daemon-specific non-MCP log rotation (ai/daemons/**).

Avoided Traps

• Do not file this as an outage: current source already has payload caps and age/count retention.
• Do not delete the active current-day file to satisfy a byte budget; that violates the existing active-stream safety invariant.
• Do not introduce magic constants beside the provider leaves. ADR 0019 makes the config provider the authority.

Related

• #10582 / #10576 — archived daily-rotation lineage; disk-bloat policy was out of scope there.
• #13480 / #13481 — wake stability work that surfaced long-running harness behavior.
• #13475 — MCP sibling-process diagnostics.
• #13506 / PR #13507 — Memory Core MCP tool telemetry.

Handoff Retrieval Hints: query_raw_memories("Neural Link payload cap MCP logs 106M nl-server loggerRetention maxTotalBytes"), query_raw_memories("MCP shared logger retention age count active current-day file preserved").