Mailbox polish: countMessages primitive + concept provenance docs
Context
Two small Phase-4-followup items surfaced during the cross-model review cycle on #10175 and #10177. Aggregated here because both are mailbox-substrate polish, neither is load-bearing for current functionality, and splitting them into separate tickets would add review-overhead proportional to review value rather than change value. Filed so knowledge doesn't get lost between the merge of #10177 and whenever the next Phase 4 polish cycle fires.
The Problem
Problem 1 — getHealthcheckPreview unread count caps at 100.
MailboxService.getHealthcheckPreview (introduced in #10177) counts unreads by fetching this.listMessages({box: 'inbox', limit: 100}) and iterating for !msg.readAt. The limit of 100 is the current effective upper bound on unreadCount — an inbox with more than 100 unread messages will under-report. Fine at current swarm volumes (3 agents, per-agent inbox typically 0–5), but the cap is silently-incorrect rather than explicitly documented.
A natural fix is a dedicated MailboxService.countMessages({status, box, fromIdentity?}) primitive that returns a scalar count via direct SQL — patterned after #10178's buildMailboxDelta, which does the same thing in MemoryService.addMemory via a SELECT COUNT(DISTINCT e.source) against the edge table. Shared primitive retires the limit-100 proxy pattern entirely.
Problem 2 — auto_extracted concept-provenance flag is undocumented.
#10175 introduced an auto_extracted: true property on CONCEPT:* and CLASS:* nodes auto-seeded during MailboxService.addMessage → SemanticGraphExtractor.extractMessageConcepts fire-and-forget. The flag distinguishes agent-manually-tagged concepts (no flag; curated) from LLM-auto-extracted concepts (flag set; provenance stamped). Downstream consumers reading concept nodes from the Native Edge Graph (DreamService, Librarian traversal, future GraphRAG query) have no way to distinguish the two populations without this signal — but the signal isn't documented anywhere outside the PR source code.
learn/agentos/ConceptOntology.md is the canonical docs home for concept-node conventions; currently silent on the auto_extracted contract.
The Architectural Reality
Problem 1 files:
ai/mcp/server/memory-core/services/MailboxService.mjs — new countMessages method; getHealthcheckPreview refactored to call it
ai/mcp/server/memory-core/services/MemoryService.mjs — buildMailboxDelta may optionally delegate to the new primitive (DRY; keep the direct-SQL fast-path to sidestep the in-memory cache coherence limitation described in #10178)
ai/mcp/server/memory-core/openapi.yaml — new count_messages tool exposed via MCP surface (or keep as internal service method; see Out of Scope)
test/playwright/unit/ai/mcp/server/memory-core/services/MailboxService.spec.mjs — regression coverage for >100 inbox depth
Problem 2 files:
learn/agentos/ConceptOntology.md — new subsection documenting auto_extracted provenance flag, the write path (addMessage → SemanticGraphExtractor.extractMessageConcepts), the edge-weight distinction (auto-extracted 0.8 vs manual 1.0), and read-time consumer guidance (distinguishing curated from auto for ranking or filtering)
ai/daemons/services/SemanticGraphExtractor.mjs — extractMessageConcepts JSDoc extended with @see link to the docs page
The Fix
- Add
MailboxService.countMessages({status, box, fromIdentity?}) returning {count: Number} via direct SQL (follow buildMailboxDelta's pattern in MemoryService.mjs — prepare + get against Edges joined with Nodes filtering on SENT_TO / SENT_BY + readAt IS NULL as appropriate).
- Refactor
getHealthcheckPreview to call countMessages for unreadCount instead of fetching 100 messages to count them. Keep limit: 3 for the preview items themselves.
- Update
learn/agentos/ConceptOntology.md with the auto_extracted provenance section (describe write path + consumer read pattern + edge-weight convention).
- Extend
SemanticGraphExtractor.extractMessageConcepts JSDoc with @see pointing to the new docs section.
Acceptance Criteria
Out of Scope
- MCP tool surface for
countMessages — start as internal service method; add MCP tool only if an external consumer (Dream daemon, Librarian, operator healthcheck dashboard) needs it. Avoids growing the tool surface for a function only getHealthcheckPreview currently calls.
- Migration of
buildMailboxDelta — #10178's direct-SQL path already works correctly; refactoring to use countMessages is a DRY concern, not a correctness concern; defer to a separate follow-up if it becomes a drift risk.
- Concurrency back-pressure on
extractMessageConcepts — separate concern; already noted in #10175 review; belongs to a dedicated ticket if bursty-traffic rate-limit issues materialize.
Avoided Traps
- "Just raise
limit to 10000." Rejected. It's still a silent cap; fetching 10000 messages to count them is pathologically expensive at scale. Direct SQL COUNT is O(indexed) for both correctness and performance.
- "Document
auto_extracted in the PR description only." Rejected. PR descriptions are ephemeral for the Retrospective daemon; canonical docs in learn/agentos/ is the A2A memory bridge downstream consumers actually query.
Related
- Parent:
#10139 Mailbox A2A primitive (epic)
- Surfaced by:
#10175 review ([KB_GAP] tag on the auto_extracted convention), #10177 review (100-cap observation on getHealthcheckPreview)
- Builds on:
#10178 (direct-SQL pattern in buildMailboxDelta is the template for countMessages)
- Adjacent:
#10172 node-ID case canonicalization (CONCEPT:/CLASS: case coordination — see #10172 comment cross-referencing this ticket)
Origin Session ID: 7ee23599-3f94-4f75-b54c-97ba92c9aaef
Mailbox polish: countMessages primitive + concept provenance docs
Context
Two small Phase-4-followup items surfaced during the cross-model review cycle on
#10175and#10177. Aggregated here because both are mailbox-substrate polish, neither is load-bearing for current functionality, and splitting them into separate tickets would add review-overhead proportional to review value rather than change value. Filed so knowledge doesn't get lost between the merge of#10177and whenever the next Phase 4 polish cycle fires.The Problem
Problem 1 —
getHealthcheckPreviewunread count caps at 100.MailboxService.getHealthcheckPreview(introduced in#10177) counts unreads by fetchingthis.listMessages({box: 'inbox', limit: 100})and iterating for!msg.readAt. The limit of 100 is the current effective upper bound onunreadCount— an inbox with more than 100 unread messages will under-report. Fine at current swarm volumes (3 agents, per-agent inbox typically 0–5), but the cap is silently-incorrect rather than explicitly documented.A natural fix is a dedicated
MailboxService.countMessages({status, box, fromIdentity?})primitive that returns a scalar count via direct SQL — patterned after#10178'sbuildMailboxDelta, which does the same thing inMemoryService.addMemoryvia aSELECT COUNT(DISTINCT e.source)against the edge table. Shared primitive retires the limit-100 proxy pattern entirely.Problem 2 —
auto_extractedconcept-provenance flag is undocumented.#10175introduced anauto_extracted: trueproperty onCONCEPT:*andCLASS:*nodes auto-seeded duringMailboxService.addMessage→SemanticGraphExtractor.extractMessageConceptsfire-and-forget. The flag distinguishes agent-manually-tagged concepts (no flag; curated) from LLM-auto-extracted concepts (flag set; provenance stamped). Downstream consumers reading concept nodes from the Native Edge Graph (DreamService, Librarian traversal, future GraphRAG query) have no way to distinguish the two populations without this signal — but the signal isn't documented anywhere outside the PR source code.learn/agentos/ConceptOntology.mdis the canonical docs home for concept-node conventions; currently silent on theauto_extractedcontract.The Architectural Reality
Problem 1 files:
ai/mcp/server/memory-core/services/MailboxService.mjs— newcountMessagesmethod;getHealthcheckPreviewrefactored to call itai/mcp/server/memory-core/services/MemoryService.mjs—buildMailboxDeltamay optionally delegate to the new primitive (DRY; keep the direct-SQL fast-path to sidestep the in-memory cache coherence limitation described in#10178)ai/mcp/server/memory-core/openapi.yaml— newcount_messagestool exposed via MCP surface (or keep as internal service method; see Out of Scope)test/playwright/unit/ai/mcp/server/memory-core/services/MailboxService.spec.mjs— regression coverage for >100 inbox depthProblem 2 files:
learn/agentos/ConceptOntology.md— new subsection documentingauto_extractedprovenance flag, the write path (addMessage→SemanticGraphExtractor.extractMessageConcepts), the edge-weight distinction (auto-extracted 0.8 vs manual 1.0), and read-time consumer guidance (distinguishing curated from auto for ranking or filtering)ai/daemons/services/SemanticGraphExtractor.mjs—extractMessageConceptsJSDoc extended with@seelink to the docs pageThe Fix
MailboxService.countMessages({status, box, fromIdentity?})returning{count: Number}via direct SQL (followbuildMailboxDelta's pattern inMemoryService.mjs—prepare+getagainstEdgesjoined withNodesfiltering onSENT_TO/SENT_BY+readAt IS NULLas appropriate).getHealthcheckPreviewto callcountMessagesforunreadCountinstead of fetching 100 messages to count them. Keeplimit: 3for the preview items themselves.learn/agentos/ConceptOntology.mdwith theauto_extractedprovenance section (describe write path + consumer read pattern + edge-weight convention).SemanticGraphExtractor.extractMessageConceptsJSDoc with@seepointing to the new docs section.Acceptance Criteria
MailboxService.countMessages({status, box, fromIdentity?})implemented with direct-SQL pathgetHealthcheckPreviewusescountMessagesforunreadCount; no longer useslimit: 100proxyunreadCountreturns the correct value (not capped)countMessagesmatcheslistMessages({status: 'unread'}).lengthfor small inboxeslearn/agentos/ConceptOntology.mdgains anauto_extractedprovenance sectionSemanticGraphExtractor.extractMessageConceptsJSDoc carries@seecross-referenceOut of Scope
countMessages— start as internal service method; add MCP tool only if an external consumer (Dream daemon, Librarian, operator healthcheck dashboard) needs it. Avoids growing the tool surface for a function onlygetHealthcheckPreviewcurrently calls.buildMailboxDelta—#10178's direct-SQL path already works correctly; refactoring to usecountMessagesis a DRY concern, not a correctness concern; defer to a separate follow-up if it becomes a drift risk.extractMessageConcepts— separate concern; already noted in#10175review; belongs to a dedicated ticket if bursty-traffic rate-limit issues materialize.Avoided Traps
limitto 10000." Rejected. It's still a silent cap; fetching 10000 messages to count them is pathologically expensive at scale. Direct SQL COUNT is O(indexed) for both correctness and performance.auto_extractedin the PR description only." Rejected. PR descriptions are ephemeral for the Retrospective daemon; canonical docs inlearn/agentos/is the A2A memory bridge downstream consumers actually query.Related
#10139Mailbox A2A primitive (epic)#10175review ([KB_GAP] tag on theauto_extractedconvention),#10177review (100-cap observation ongetHealthcheckPreview)#10178(direct-SQL pattern inbuildMailboxDeltais the template forcountMessages)#10172node-ID case canonicalization (CONCEPT:/CLASS: case coordination — see#10172comment cross-referencing this ticket)Origin Session ID:
7ee23599-3f94-4f75-b54c-97ba92c9aaef