LearnNewsExamplesServices
Frontmatter
titlefeat(ai): add scheduled MX concept harvester (#13840)
authorneo-gpt
stateMerged
createdAtJun 24, 2026, 10:46 PM
updatedAtJun 24, 2026, 10:55 PM
closedAtJun 24, 2026, 10:55 PM
mergedAtJun 24, 2026, 10:55 PM
branchesdevcodex/13840-mx-concept-harvester
urlhttps://github.com/neomjs/neo/pull/13968
contentTrust
projected
quarantined0
signals[]
Merged
neo-gpt
neo-gpt commented on Jun 24, 2026, 10:46 PM

Resolves #13840

Adds a scheduled process/MX concept harvester for A2A MESSAGE nodes: it scans unharvested messages, builds a cheap subject/tag frequency pre-filter, sends only the bounded top terms through the existing Teaching-Test extractor, dedupes by normalized concept name, writes process/MX candidates, and marks source messages only after provider evaluation and candidate append succeed.

Evidence: L2 local unit/static checks -> L2 required (scheduled drainer, config leaves, graph projection, and scheduler behavior are unit-testable). Residual: none for #13840.

Deltas from ticket

  • Keeps process/MX terms in the existing concept ontology but tags them with ontologyLayer: 'process-mx' and codeGapEligible: false, so they can enrich MX/coordination knowledge without polluting code-doc gap inference.
  • Adds strict drainer semantics: provider/parser failures and append failures fail the scheduled task and do not stamp messages as harvested.
  • Adds deterministic normalized-name dedupe for mined candidates, covering punctuation/case/acronym variants before JSONL append.

Config / Local Overlay Follow-Up

Changed committed config-template keys:

  • orchestrator.intervals.messageConceptHarvestMs / NEO_ORCHESTRATOR_MESSAGE_CONCEPT_HARVEST_INTERVAL_MS (default: 6h)
  • conceptDiscovery.messageHarvestBatchLimit / NEO_CONCEPT_DISCOVERY_MESSAGE_HARVEST_BATCH_LIMIT (default: 500)
  • conceptDiscovery.messageHarvestTopN / NEO_CONCEPT_DISCOVERY_MESSAGE_HARVEST_TOP_N (default: 20)
  • conceptDiscovery.messageHarvestMinFrequency / NEO_CONCEPT_DISCOVERY_MESSAGE_HARVEST_MIN_FREQUENCY (default: 2)

Matching local config.mjs overlays need the standard post-merge config overlay update script. Harness/orchestrator restart is recommended after that so the new scheduler and Memory Core leaves are loaded by live daemons.

Test Evidence

  • npm run agent-preflight -- <touched files> passed.
  • git diff --check origin/dev..HEAD passed.
  • npm run test-unit -- test/playwright/unit/ai/services/ingestion/ConceptDiscoveryService.spec.mjs test/playwright/unit/ai/services/ingestion/ConceptIngestor.spec.mjs test/playwright/unit/ai/daemons/orchestrator/services/DreamService.spec.mjs test/playwright/unit/ai/daemons/orchestrator/scheduling/registry.spec.mjs test/playwright/unit/ai/daemons/orchestrator/scheduling/pipeline.spec.mjs test/playwright/unit/ai/daemons/orchestrator/services/MaintenanceBackpressureService.spec.mjs test/playwright/unit/ai/daemons/orchestrator/Orchestrator.spec.mjs test/playwright/unit/ai/config.template.spec.mjs test/playwright/unit/ai/mcp/server/memory-core/config.template.spec.mjs passed: 195 tests.

Post-Merge Validation

  • Run node ./ai/scripts/setup/initServerConfigs.mjs --migrate-config.
  • Restart orchestrator/harness and verify message-concept-harvest appears in task state after the first due poll.
  • Confirm process/MX harvested candidates do not emit code-doc GUIDE_GAP / EXAMPLE_GAP / ORPHAN_CONCEPT findings.

Commits

  • 30aa395cacfeat(ai): add scheduled MX concept harvester (#13840)

Authored by Euclid (GPT-5, Codex Desktop). Session 30ba1931-9d59-48bb-afdf-af8904070af9.