Skip to content

fix(infra): swallow mid-stream read errors in session-cost readJsonlRecords - #101062

Merged
vincentkoc merged 3 commits into
openclaw:mainfrom
cxbAsDev:fix/session-cost-usage-readline-errors
Jul 7, 2026
Merged

fix(infra): swallow mid-stream read errors in session-cost readJsonlRecords#101062
vincentkoc merged 3 commits into
openclaw:mainfrom
cxbAsDev:fix/session-cost-usage-readline-errors

Conversation

@cxbAsDev

@cxbAsDev cxbAsDev commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

What Problem This Solves

readJsonlRecords reads session transcript JSONL through a readline interface backed by fs.createReadStream. A mid-stream read failure can reject diagnostic session-log reads. Swallowing that failure in the shared reader is unsafe, though, because the same reader feeds the durable usage cache and could persist a partial scan as a fresh complete entry.

Why This Change Was Made

Keep the shared JSONL reader strict so cache scans fail before replacing the durable entry. Add a best-effort wrapper only for loadSessionLogs, where returning the records read before a stream failure is appropriate diagnostic behavior.

The regression coverage also verifies that a failed append-only cache refresh rejects, leaves the previous cache bytes unchanged, and reports the changed transcript as pending instead of fresh.

User Impact

Session-log diagnostics no longer surface raw mid-stream read errors, while incomplete usage scans cannot silently poison the durable usage cache.

Evidence

  • OPENCLAW_LOCAL_CHECK_MODE=throttled node scripts/run-vitest.mjs src/infra/session-cost-usage.stream-errors.test.ts src/infra/session-cost-usage.test.ts — 67 tests passed.
  • OPENCLAW_LOCAL_CHECK_MODE=throttled node_modules/.bin/tsx scripts/proof/session-cost-usage-readline-errors.mts — EISDIR proof passed.
  • OPENCLAW_LOCAL_CHECK_MODE=throttled OPENCLAW_CHECK_CHANGED_REMOTE_CHILD=1 OPENCLAW_CHANGED_LANES_RAW_SYNC=1 PNPM_CONFIG_VERIFY_DEPS_BEFORE_RUN=false pnpm check:changed — passed locally; no Testbox/Blacksmith used.
  • codex review --base origin/main — no actionable findings.

@openclaw-barnacle openclaw-barnacle Bot added scripts Repository scripts size: S labels Jul 6, 2026
@cxbAsDev

cxbAsDev commented Jul 6, 2026

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

@cxbAsDev
cxbAsDev force-pushed the fix/session-cost-usage-readline-errors branch 2 times, most recently from dc1de00 to 79b03b5 Compare July 6, 2026 17:59
@clawsweeper

clawsweeper Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge. Reviewed July 6, 2026, 9:42 PM ET / 01:42 UTC.

Summary
The PR adds a diagnostic-only best-effort JSONL reader for loadSessionLogs, leaves durable session-cost cache scans strict, and adds regression/proof coverage for stream read errors.

PR surface: Source +11, Tests +99, Other +47. Total +157 across 3 files.

Reproducibility: yes. at source level: current main has loadSessionLogs iterating readJsonlRecords directly, and a Node readline probe confirms input stream errors reject async iteration.

Review metrics: none identified.

Stored data model
Persistent data-model change detected: persistent cache schema: src/infra/session-cost-usage.stream-errors.test.ts, serialized state: scripts/proof/session-cost-usage-readline-errors.mts, serialized state: src/infra/session-cost-usage.stream-errors.test.ts, serialized state: src/infra/session-cost-usage.ts, unknown-data-model-change: scripts/proof/session-cost-usage-readline-errors.mts, unknown-data-model-change: src/infra/session-cost-usage.stream-errors.test.ts. Confirm migration or upgrade compatibility proof before merge.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🐚 platinum hermit
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • none.

Risk before merge

  • [P1] The live PR is mergeable but behind current main; the touched files did not change after its base, so this is an exact-head validation/rebase risk rather than a code defect.

Maintainer options:

  1. Decide the mitigation before merge
    Land the diagnostic-only best-effort reader after exact-head merge validation, keeping durable cache scans strict on stream read failures.
  2. Pause or close
    Do not merge this PR until maintainers decide whether the risk is worth taking.

Next step before merge

  • [P2] No repair lane is needed; the PR has no actionable findings and only needs normal maintainer merge validation.

Security
Cleared: The diff adds no dependencies, workflow changes, credential handling, network access, or new supply-chain execution path beyond a local proof script.

Review details

Best possible solution:

Land the diagnostic-only best-effort reader after exact-head merge validation, keeping durable cache scans strict on stream read failures.

Do we have a high-confidence way to reproduce the issue?

Yes, at source level: current main has loadSessionLogs iterating readJsonlRecords directly, and a Node readline probe confirms input stream errors reject async iteration.

Is this the best way to solve the issue?

Yes: catching only at loadSessionLogs is the narrow diagnostic-boundary fix; swallowing in the shared reader would risk partial durable cache scans being marked fresh.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 253b784468c6.

Label changes

Label justifications:

  • P2: The PR is a focused reliability fix for session usage/log diagnostics with limited blast radius.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): Sufficient: the PR body and maintainer comment report focused tests plus terminal EISDIR proof, and the included proof script exercises the real stream-error path.
  • proof: sufficient: Contributor real behavior proof is sufficient. Sufficient: the PR body and maintainer comment report focused tests plus terminal EISDIR proof, and the included proof script exercises the real stream-error path.
Evidence reviewed

PR surface:

Source +11, Tests +99, Other +47. Total +157 across 3 files.

View PR surface stats
Area Files Added Removed Net
Source 1 12 1 +11
Tests 1 99 0 +99
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 1 47 0 +47
Total 3 158 1 +157

What I checked:

Likely related people:

  • steipete: Git blame on the current main reader, cache scan, and diagnostic load path points to commit 66c764b269357417aa8950b5dea92becf6f3a64f, which is PR feat(ios): tap-to-expand link previews in chat transcript #101198 authored by steipete. (role: introduced current session-cost usage surface; confidence: high; commits: 66c764b26935; files: src/infra/session-cost-usage.ts, src/infra/session-cost-usage.test.ts)
  • vincentkoc: Live PR metadata and comments show vincentkoc is assigned and pushed the current repair commit that keeps cache scans strict while narrowing best-effort behavior to diagnostics. (role: recent repair author and assignee; confidence: high; commits: f4df8861c00d, aa89aa6e495c; files: src/infra/session-cost-usage.ts, src/infra/session-cost-usage.stream-errors.test.ts, scripts/proof/session-cost-usage-readline-errors.mts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.
Review history (5 earlier review cycles)
  • reviewed 2026-07-06T18:44:25.089Z sha 79b03b5 :: needs maintainer review before merge. :: none
  • reviewed 2026-07-06T21:49:22.595Z sha aa89aa6 :: needs maintainer review before merge. :: none
  • reviewed 2026-07-06T23:58:20.201Z sha aa89aa6 :: needs maintainer review before merge. :: none
  • reviewed 2026-07-07T01:08:35.294Z sha 95dcf76 :: needs maintainer review before merge. :: none
  • reviewed 2026-07-07T01:17:14.117Z sha 95dcf76 :: needs maintainer review before merge. :: none

@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. P2 Normal backlog priority with limited blast radius. labels Jul 6, 2026
@vincentkoc vincentkoc self-assigned this Jul 6, 2026
@vincentkoc
vincentkoc force-pushed the fix/session-cost-usage-readline-errors branch from 79b03b5 to aa89aa6 Compare July 6, 2026 21:28
@openclaw-barnacle openclaw-barnacle Bot added triage: blank-template Candidate: PR template appears mostly untouched. triage: needs-pr-context Candidate: external PR body lacks required problem context or evidence. labels Jul 6, 2026
@vincentkoc

Copy link
Copy Markdown
Member

Maintainer repair pushed at aa89aa6e495c54e39b4c8205b4f0c0185456b481 and rebased onto current origin/main (c7295e417d5daec76c18fb452d117f7b8eadc4d6). Contributor authorship is preserved in the rebased source commit.

The shared JSONL reader is strict again, so a failed durable-cache scan cannot stamp partial usage as fresh. loadSessionLogs alone uses the best-effort reader, preserving the intended diagnostic behavior. Added regression coverage proves the previous cache bytes remain unchanged and the transcript stays pending after a stream failure.

Validation:

  • focused session-cost tests: 67 passed
  • EISDIR real-behavior proof: passed
  • local throttled pnpm check:changed: passed
  • codex review --base origin/main: no actionable findings
  • no Blacksmith/Testbox used

@clawsweeper review

@clawsweeper

clawsweeper Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

@openclaw-barnacle openclaw-barnacle Bot removed the triage: needs-pr-context Candidate: external PR body lacks required problem context or evidence. label Jul 6, 2026
@vincentkoc

Copy link
Copy Markdown
Member

No description provided.

@vincentkoc
vincentkoc force-pushed the fix/session-cost-usage-readline-errors branch from 95dcf76 to f4df886 Compare July 7, 2026 01:22
@cxbAsDev

cxbAsDev commented Jul 7, 2026

Copy link
Copy Markdown
Contributor Author

Description has been filled in. The PR body now includes the problem, rationale, user impact, and evidence sections. Please let me know if anything else is needed.

@vincentkoc

Copy link
Copy Markdown
Member

No description provided.

@vincentkoc

Copy link
Copy Markdown
Member

No description provided.

@vincentkoc
vincentkoc merged commit 325cdb7 into openclaw:main Jul 7, 2026
101 checks passed
github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request Jul 7, 2026
…ecords (openclaw#101062)

* fix(infra): swallow mid-stream read errors in session-cost readJsonlRecords

* fix(infra): keep usage cache strict on read errors

---------

Co-authored-by: Vincent Koc <25068+vincentkoc@users.noreply.github.com>
Co-authored-by: Vincent Koc <vincentkoc@ieee.org>
sheyanmin pushed a commit to sheyanmin/openclaw that referenced this pull request Jul 8, 2026
…ecords (openclaw#101062)

* fix(infra): swallow mid-stream read errors in session-cost readJsonlRecords

* fix(infra): keep usage cache strict on read errors

---------

Co-authored-by: Vincent Koc <25068+vincentkoc@users.noreply.github.com>
Co-authored-by: Vincent Koc <vincentkoc@ieee.org>
giodl73-repo pushed a commit to giodl73-repo/openclaw that referenced this pull request Jul 8, 2026
…ecords (openclaw#101062)

* fix(infra): swallow mid-stream read errors in session-cost readJsonlRecords

* fix(infra): keep usage cache strict on read errors

---------

Co-authored-by: Vincent Koc <25068+vincentkoc@users.noreply.github.com>
Co-authored-by: Vincent Koc <vincentkoc@ieee.org>
@cxbAsDev
cxbAsDev deleted the fix/session-cost-usage-readline-errors branch July 15, 2026 03:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

P2 Normal backlog priority with limited blast radius. proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. scripts Repository scripts size: S status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. triage: blank-template Candidate: PR template appears mostly untouched.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants