Context
#12329 established that GraphLog (the Native Edge Graph CDC table) grows monotonically and unbounded during normal operation — the live production read in #12329 found it at 7.3M rows / 365 MB (83% of the graph db). #12337 delivered the compaction mechanism: ai/scripts/maintenance/compactGraphLog.mjs, a dry-run-first CLI that prunes rows at/below the minimum live-consumer watermark (#12389 fixes its standalone-Node bootstrap).
But that is a manual CLI. #12329's ACs delivered the mechanism (AC2: "a compaction mechanism … runnable periodically"); none of them schedules it. So the CDC log only stays bounded if a human remembers to run the CLI — and resumes unbounded growth between runs. For a continuously-appended CDC table, that is the recurring-job-shipped-as-a-one-shot gap.
The Problem
GraphLog compaction is recurring maintenance with no automation. The Agent OS already owns a periodic-maintenance scheduler — the Orchestrator (ADR 0014) — which runs sibling store-maintenance lanes (backup, chromaDefrag, summary, dream, golden-path). GraphLog compaction is the missing sqlite-graph-store equivalent of the existing chromaDefrag (chroma-store) lane. Cloud deployments are more exposed, not less: their orchestrator runs the graph-mutating dream/summary lanes, so their GraphLog grows the same way, with no operator at a shell to run the CLI.
The Architectural Reality
- Scheduler substrate:
ai/daemons/orchestrator/TaskDefinitions.mjs (buildTaskDefinitions — the lane→child-process command map; e.g. backup → maintenance/backup.mjs) + ai/daemons/orchestrator/scheduling/*.mjs (per-lane getDueTask cadence) + Orchestrator.mjs (the lane registry + cloud-safe deployment-policy classification, e.g. tenant-repo-sync cloud-deployable vs primary-dev-sync local-only).
- The job to invoke:
ai/scripts/maintenance/compactGraphLog.mjs (--apply; the watermark-safety contract — never prune above min(live-consumer watermark) minus DEFAULT_SAFETY_MARGIN_ROWS — already lives in the CLI per #12329 AC3, so the lane delegates safety to it).
- Precedents to mirror:
backup (low-frequency, cloud-deployable, retention-bounded) and chromaDefrag (the chroma store-maintenance lane). This ticket adds the symmetric graphlog-compaction lane.
The Fix
Add a graphlog-compaction Orchestrator lane:
TaskDefinitions.mjs — a lane entry invoking maintenance/compactGraphLog.mjs --apply (with --vacuum per cadence/config), pidfile, expected-command guard, mirroring backup.
scheduling/graphLogCompaction.mjs — a getDueTask with a low-frequency cadence (config-driven, default daily/weekly like backup).
Orchestrator.mjs — register the lane + classify it in the cloud-safe deployment policy (it should run in cloud deployments — their GraphLog grows too).
- Cadence/enable knobs as config leaves (sibling to the backup retention config).
Contract Ledger
| Target Surface |
Source of Authority |
Proposed Behavior |
Fallback |
Docs |
Evidence |
TaskDefinitions.buildTaskDefinitions graphlog-compaction lane |
this ticket |
child-process command invoking compactGraphLog.mjs --apply |
absent → no scheduled compaction (today's state) |
TaskDefinitions JSDoc |
Orchestrator.spec lane-definition assertion |
scheduling/graphLogCompaction.mjs getDueTask |
this ticket + scheduling/backup.mjs precedent |
low-frequency due-check (config cadence) |
unset cadence → conservative default |
scheduling JSDoc |
unit: due/not-due across the cadence window |
| Cloud-safe deployment-policy classification |
ADR 0014 + Orchestrator.mjs policy |
graphlog-compaction runs in cloud deployments |
local-only opt-out if a deployment disables it |
deployment-policy comment |
Orchestrator.spec classification assertion |
| Watermark-prune safety |
compactGraphLog.mjs (#12329 AC3) |
unchanged — delegated to the CLI |
the CLI fail-closes on unknown watermarks |
n/a (existing) |
existing compactGraphLog.spec |
Decision Record impact
aligned-with ADR 0014 (Cloud Deployment Topology + Scheduler Task Taxonomy) — extends the orchestrator lane taxonomy with the symmetric graph-store maintenance lane; does not challenge it.
Acceptance Criteria
Out of Scope
- The compaction mechanism itself (
compactGraphLog.mjs) — delivered by #12329 / #12337; bootstrap by #12389.
- The one-time existing-backlog purge (operator-gated; #12329 AC5).
- The upstream mutation-churn source (REM-cycle /
GapInferenceEngine tuning) — a separate concern noted on #12329.
Related
- #12329 (GraphLog unbounded-growth root + the compaction mechanism) · #12337 (CLI) · #12389 (CLI bootstrap fix — this lane is blocked by the CLI booting in a standalone Node process).
chromaDefrag / backup orchestrator lanes (the maintenance-lane precedent).
Origin Session ID: c279991f-11eb-4e89-9e38-2c9c4a78421e
Handoff Retrieval Hints: query_raw_memories("GraphLog compaction orchestrator lane recurring maintenance chromaDefrag precedent"); anchors ai/daemons/orchestrator/TaskDefinitions.mjs, ai/scripts/maintenance/compactGraphLog.mjs.
Authored by @neo-opus-ada (claude-opus-4.8-1m)
Context
#12329 established that
GraphLog(the Native Edge Graph CDC table) grows monotonically and unbounded during normal operation — the live production read in #12329 found it at 7.3M rows / 365 MB (83% of the graph db). #12337 delivered the compaction mechanism:ai/scripts/maintenance/compactGraphLog.mjs, a dry-run-first CLI that prunes rows at/below the minimum live-consumer watermark (#12389 fixes its standalone-Node bootstrap).But that is a manual CLI. #12329's ACs delivered the mechanism (AC2: "a compaction mechanism … runnable periodically"); none of them schedules it. So the CDC log only stays bounded if a human remembers to run the CLI — and resumes unbounded growth between runs. For a continuously-appended CDC table, that is the recurring-job-shipped-as-a-one-shot gap.
The Problem
GraphLog compaction is recurring maintenance with no automation. The Agent OS already owns a periodic-maintenance scheduler — the Orchestrator (ADR 0014) — which runs sibling store-maintenance lanes (
backup,chromaDefrag,summary,dream,golden-path). GraphLog compaction is the missing sqlite-graph-store equivalent of the existingchromaDefrag(chroma-store) lane. Cloud deployments are more exposed, not less: their orchestrator runs the graph-mutatingdream/summarylanes, so their GraphLog grows the same way, with no operator at a shell to run the CLI.The Architectural Reality
ai/daemons/orchestrator/TaskDefinitions.mjs(buildTaskDefinitions— the lane→child-process command map; e.g.backup→maintenance/backup.mjs) +ai/daemons/orchestrator/scheduling/*.mjs(per-lanegetDueTaskcadence) +Orchestrator.mjs(the lane registry + cloud-safe deployment-policy classification, e.g.tenant-repo-synccloud-deployable vsprimary-dev-synclocal-only).ai/scripts/maintenance/compactGraphLog.mjs(--apply; the watermark-safety contract — never prune abovemin(live-consumer watermark)minusDEFAULT_SAFETY_MARGIN_ROWS— already lives in the CLI per #12329 AC3, so the lane delegates safety to it).backup(low-frequency, cloud-deployable, retention-bounded) andchromaDefrag(the chroma store-maintenance lane). This ticket adds the symmetricgraphlog-compactionlane.The Fix
Add a
graphlog-compactionOrchestrator lane:TaskDefinitions.mjs— a lane entry invokingmaintenance/compactGraphLog.mjs --apply(with--vacuumper cadence/config), pidfile, expected-command guard, mirroringbackup.scheduling/graphLogCompaction.mjs— agetDueTaskwith a low-frequency cadence (config-driven, default daily/weekly likebackup).Orchestrator.mjs— register the lane + classify it in the cloud-safe deployment policy (it should run in cloud deployments — their GraphLog grows too).Contract Ledger
TaskDefinitions.buildTaskDefinitionsgraphlog-compactionlanecompactGraphLog.mjs --applyOrchestrator.speclane-definition assertionscheduling/graphLogCompaction.mjsgetDueTaskscheduling/backup.mjsprecedentOrchestrator.mjspolicygraphlog-compactionruns in cloud deploymentsOrchestrator.specclassification assertioncompactGraphLog.mjs(#12329 AC3)compactGraphLog.specDecision Record impact
aligned-with ADR 0014(Cloud Deployment Topology + Scheduler Task Taxonomy) — extends the orchestrator lane taxonomy with the symmetric graph-store maintenance lane; does not challenge it.Acceptance Criteria
graphlog-compactionOrchestrator lane exists (TaskDefinitions entry +scheduling/getDueTask) that periodically invokescompactGraphLog.mjs --apply.Orchestrator.spec).Orchestrator.spec; due/not-due unit for the new scheduler.Out of Scope
compactGraphLog.mjs) — delivered by #12329 / #12337; bootstrap by #12389.GapInferenceEnginetuning) — a separate concern noted on #12329.Related
chromaDefrag/backuporchestrator lanes (the maintenance-lane precedent).Origin Session ID: c279991f-11eb-4e89-9e38-2c9c4a78421e
Handoff Retrieval Hints:
query_raw_memories("GraphLog compaction orchestrator lane recurring maintenance chromaDefrag precedent"); anchorsai/daemons/orchestrator/TaskDefinitions.mjs,ai/scripts/maintenance/compactGraphLog.mjs.Authored by @neo-opus-ada (claude-opus-4.8-1m)