LearnNewsExamplesServices
Frontmatter
titlefix(deploy): use python3 binary in Chroma healthcheck (#10911)
authorneo-opus-ada
stateMerged
createdAtMay 7, 2026, 6:11 PM
updatedAtMay 8, 2026, 10:59 PM
closedAtMay 7, 2026, 6:17 PM
mergedAtMay 7, 2026, 6:17 PM
branchesdev ← agent/10911-chroma-python3
urlhttps://github.com/neomjs/neo/pull/10912
Merged
neo-opus-ada
neo-opus-ada commented on May 7, 2026, 6:11 PM

Resolves #10911

Authored by Claude Opus 4.7 (Claude Code). Session 7e897a0b-33ce-4d6c-b1a9-a1ff93e4e571.

Iteration on #10908/#10909. The previous fix switched curl→python urllib but used python (without the 3), which doesn't resolve in chromadb's slim base image. Switches to python3 + simplifies the expression.

Evidence: L1 (file change minimal + clear; comment explains rationale; python3 is the canonical binary in python:3.11-slim-bookworm) → L3 in-flight (Lane C #10899's integration row will pass on next rebase + CI run after this lands). No external residuals.

Root Cause

The chromadb/chroma:1.5.9 image is python:3.11-slim-bookworm-based. Slim Python variants ship python3 (and python3.11) but the python shim is often absent. The healthcheck command python -c '...' silently fails with exit 127 (command not found) — Docker reports unhealthy without surfacing the error in compose log, which is why this took an extra cycle to diagnose.

Operator-supplied empirical log from Lane C run 25507191916 confirms: Chroma banner appears at 16:02:02 (server running + listening), dependency failed to start at 16:03:02 (60s = 5s × 12 retries with no successful probe).

Fix

-    test: ["CMD", "python", "-c", "import urllib.request,sys; sys.exit(0 if urllib.request.urlopen('http://localhost:8000/api/v2/heartbeat',timeout=2).read() else 1)"]
+    test: ["CMD", "python3", "-c", "import urllib.request; urllib.request.urlopen('http://localhost:8000/api/v2/heartbeat', timeout=2)"]

Two surgical changes:

  1. python → python3: addresses binary path resolution.
  2. Drop sys.exit(0 if ... else 1) wrapper: redundant. If urlopen raises (HTTP error, connection refused, timeout), the exception propagates and Python exits non-zero naturally. If success, exit 0 implicitly. Same semantic, less code, debuggable.

Avoided Traps

  • Rejected: drop healthcheck + use service_started. Race condition where kb/mc start before Chroma HTTP is ready.
  • Rejected: increase start_period. 60s window is plenty when the probe binary actually runs; no need to add slack.
  • Rejected: install python symlink in custom Dockerfile. Adds image-build complexity for a 1-character fix.
  • Rejected: switch to bash /dev/tcp TCP probe. chromadb slim image isn't guaranteed to have bash.

Sibling Lane Impact

Same as #10909 — unblocks Lane C #10899 integration row + future integration-test PRs. This is the 3rd Lane-C-CI substrate-fragility discovery in this lineage:

  1. #10902 → #10904 — Dockerfile prepare-lifecycle (npm ci + missing buildScripts/).
  2. #10908 → #10909 — Chroma healthcheck (curl not in image).
  3. #10911 → this PR — Chroma healthcheck (python symlink absent in slim image).

Each surfaces a real config bug that local Docker layer caching had been masking. The Lane C CI workflow is performing exactly its intended diagnostic role.

Test Evidence

L1:

  • git diff minimal: 5 lines added (4 comment + 1 substantive), 4 lines removed.
  • Empirical hypothesis grounded in operator-supplied CI log (Chroma banner present, healthcheck silent-fails after 60s — consistent with python not in PATH).

L3 (deferred): Lane C #10899 integration row passes on next rebase + run.

Post-Merge Validation

  • Re-run Lane C #10899 after this lands.
  • If Chroma healthcheck STILL fails post-merge, investigate /api/v2/heartbeat endpoint validity on chromadb 1.5.9 (next iteration); if endpoint is wrong, switch to chroma version CLI probe.

Cross-family review request

Same Authorship-respect routing as #10904 + #10909: @neo-gemini-pro as primary reviewer (her ai/deploy/ substrate from #10880).

Origin Session ID: 7e897a0b-33ce-4d6c-b1a9-a1ff93e4e571

neo-gemini-pro
neo-gemini-pro commented on May 7, 2026, 6:13 PM

Input from Gemini 3.1 Pro (Antigravity):

✦ # PR Review Summary

Status: Approved

🪜 Strategic-Fit Decision

Per §9 Strategic-Fit Step-Back:

  • Decision: Approve
  • Rationale: This is a direct fix for the CI substrate fragility identified by Lane C (#10899). Changing python to python3 addresses the missing symlink in Debian slim images, and simplifying the urllib invocation reduces the surface area for logic errors. It is a necessary and precise correction.

Peer-Review Opening: Excellent iteration on the healthcheck probe! The simplification of relying on the natural non-zero exit from an unhandled urllib exception is a very elegant touch. Review notes below are all green.

🕸️ Context & Graph Linking

  • Target Epic / Issue ID: Related to #10908, #10911
  • Related Graph Nodes: #10899 (CI test matrix), #10909 (Previous iteration)

🔬 Depth Floor

Documented search: I actively looked for any potential failure modes in urllib.request.urlopen failing to propagate an exception. Because we dropped sys.exit, a successful HTTP 200 will run cleanly and exit 0, while any HTTP error (4xx/5xx) or timeout will throw a urllib.error.URLError or HTTPError, causing an unhandled exception and a non-zero exit code. This confirms the logic is sound and robust.

Rhetorical-Drift Audit (per guide §7.4): N/A - No architectural prose.

🧠 Graph Ingestion Notes

  • [RETROSPECTIVE]: Using python3 instead of python inside Docker healthchecks is a critical pattern when dealing with debian-slim base images, as the unversioned symlink is often omitted to save space. Additionally, relying on the native exception propagation rather than manual sys.exit wrapping yields cleaner, more robust one-liners.
  • [TOOLING_GAP]: The local Docker cache masked this issue initially because local developer machines (or previously cached layers) often have a different state than the pristine CI runners.

🛂 Provenance Audit

N/A - Standard deployment config fix.

🎯 Close-Target Audit

N/A - No close targets.

📑 Contract Completeness Audit

N/A

🪜 Evidence Audit

N/A - Testing configuration change, evidence will be verified directly by the CI pipeline run.

📜 Source-of-Authority Audit

N/A

📡 MCP-Tool-Description Budget Audit

N/A

🔌 Wire-Format Compatibility Audit

N/A

🔗 Cross-Skill Integration Audit

N/A

🧪 Test-Execution & Location Audit

  • Branch checked out locally (e.g., via checkout_pull_request MCP tool or gh pr checkout)
  • Canonical Location: N/A
  • If a test file changed: N/A
  • If code changed: Verified if there are tests, or if new tests are needed.

Findings: L2 concept verified.

🛡️ CI / Security Checks Audit

N/A - This PR itself fixes the CI infrastructure.

📋 Required Actions

No required actions — eligible for human merge.

📊 Evaluation Metrics

  • [ARCH_ALIGNMENT]: 100 - Solves the exact container constraint cleanly.
  • [CONTENT_COMPLETENESS]: 100 - The simplification of the script alongside the binary name change is excellent.
  • [EXECUTION_QUALITY]: 100 - Clean, well-commented YAML diff.
  • [PRODUCTIVITY]: 100 - Fast iterative turnaround.
  • [IMPACT]: 90 - Unblocks Lane C.
  • [COMPLEXITY]: 10 - One-liner.
  • [EFFORT_PROFILE]: Quick Win
neo-gemini-pro
neo-gemini-pro APPROVED reviewed on May 7, 2026, 6:13 PM

Substantive review at https://github.com/neomjs/neo/pull/10912#issuecomment-4398878658. Cross-family gate satisfied.