Nothing measures how much embedding work is spent re-embedding the same content

Context

#16780 needs a bounded observation of repeated batch-embedding work. Provider activity says which stage consumes inference and WAL drain says how much work remains; neither records content identity. This ticket adds that identity dimension without storing corpus text or turning the observation into an alarm.

Architectural Reality

Knowledge Base and Memory Core run in separate processes, so a TextEmbeddingService-local window cannot be read by the Memory Core metrics observer. Both recorder services already open the same plane-anchored graph SQLite. The identity observation therefore belongs in that shared artifact.

The producer is TextEmbeddingService.embedTexts(), not interactive embedText(). It records logical batch-input admissions only after provider and abort validation and before provider settlement. Pre-aborted, unsupported, empty, yielded-before-next-chunk, and locally invalid work records nothing; work that fails after admission remains observable. Transport retry attempts are not multiplied into new logical submissions.

Fix

• Replace the process-local window with shared embedding_identity_log and embedding_identity_state tables.
• Store only a recorder-owned source, timestamp, opaque id, and 128-bit SHA-256 fingerprint; never raw input.
• Bound retention to 2,048 rows and preserve durable coverage_started_at and evicted_through boundaries.
• Read watermark, retained boundary, and counts in one SQLite snapshot so concurrent eviction cannot create a partial-as-total result.
• Query the exact get_memory_core_tool_metrics().sinceMs interval; old repeats cannot contaminate a current interval.
• Publish status, reason, coverageStartedAt, oldestRetainedAt, distinct, ratio, submissions, and truncated on every response arm.
• Keep the ratio observational: submissions / distinct, and null for an observed interval with no batch submissions. A value above 1 may be legitimate and neither proves a loop nor disproves convergence.

Contract Ledger

Acceptance Criteria

• An all-unique synthetic batch yields ratio 1; this is a no-repeat control, not a convergence verdict.
• Deliberately repeated content moves the observational ratio above 1.
• The projection emits no alarm or verdict field; legitimate duplicate content remains allowed.
• An observed interval with no batch submissions returns ratio: null, never a synthetic clean value.
• Coverage start and eviction watermark make partial intervals report truncated: true, with complete controls for later intervals.
• Identity is content-derived, cross-process-visible, and stores no raw corpus text.
• The Memory Core metrics observer reads Knowledge Base and Memory Core submissions from shared SQLite over its exact sinceMs, with the complete required shape on every status arm.
• #16780 AC-1: zero chunks issue zero provider submissions, proved at the provider boundary.

Out of Scope

Automated alarm thresholds, loop adjudication or remediation, interactive single-text embedding calls, and raw-content retention.

Related

#16780, #16835, #16861.

Origin Session ID: 87f453f9-aa80-4487-9ed1-b5d91e052c43

Authored by Ada (Claude Opus 5, Claude Code); cross-process repair by @neo-gpt.