Skip to content

fix(agents): keep long-context tool-result prompts cache-stable - #95624

Merged
vincentkoc merged 13 commits into
mainfrom
fix/stable-tool-result-prompt-cache
Jun 22, 2026
Merged

fix(agents): keep long-context tool-result prompts cache-stable#95624
vincentkoc merged 13 commits into
mainfrom
fix/stable-tool-result-prompt-cache

Conversation

@vincentkoc

Copy link
Copy Markdown
Member

What Problem This Solves

Fixes an issue where long-running agent sessions on large-context models would lose the cached conversation prefix and rewrite hundreds of thousands of tokens on every turn once accumulated tool-result text crossed the aggregate prompt cap. The system prompt cache survived, but the conversation cache thrashed during active source-gather phases.

Why This Change Was Made

Prompt assembly now keeps a per-attempt, byte-stable projection of cacheable tool-result history instead of repeatedly truncating already-projected results. Aggregate accounting includes frozen projected bytes, reductions apply only to new eligible results, and the fallback clears new result text when the aggregate budget cannot otherwise be met. Projection reuse is occurrence-safe for duplicate identities and refuses to overwrite content transformed by history/media filters. The operator-configurable agents.defaults.contextLimits.toolResultMaxChars ceiling is raised to 1,000,000 as an additional mitigation for 1M-context models, with matching docs.

User Impact

Long-context sessions retain incremental prompt caching as tool results accumulate, reducing cache-write thrash and avoidable latency/cost. Existing session data is not rewritten by the prompt-only projection path, and transformed or filtered history remains authoritative.

Evidence

  • 98 focused agent regression tests pass, including repeated-build cache stability, provider-boundary behavior, duplicate identity handling, image/media-filter preservation, aggregate fallback, and context-engine integration.
  • 35 config regression tests pass, including acceptance of toolResultMaxChars: 1_000_000.
  • node scripts/check-database-first-legacy-stores.mjs passes.
  • git diff --check passes.
  • Final autoreview: autoreview clean: no accepted/actionable findings reported.
  • The broad OPENCLAW_TESTBOX=1 pnpm check:changed attempt completed core typecheck/lint lanes but the remote Testbox stopped during the database-first guard; the same guard passes locally. This is an environment/runner limitation, not a reported test failure.

This PR was prepared with AI assistance and reviewed against the repository maintainer workflow.

@openclaw-barnacle openclaw-barnacle Bot added agents Agent runtime and tooling size: M maintainer Maintainer-authored PR labels Jun 22, 2026
@clawsweeper

clawsweeper Bot commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge. Reviewed June 21, 2026, 9:44 PM ET / 01:44 UTC.

Summary
The branch adds per-attempt memoized tool-result prompt projections in the embedded runner, wires that state through prompt-history and provider transforms, and expands regression coverage for stable projections.

PR surface: Source +217, Tests +119. Total +336 across 4 files.

Reproducibility: yes. at source level: current main rebuilds aggregate tool-result projections from the evolving message set and spends reductions on older entries first. I did not run a live provider cache trace in this read-only review.

Review metrics: 2 noteworthy metrics.

  • Projection State Wiring: 1 per-attempt state added, 2 live prompt call sites wired. Prompt-cache stability depends on the pre-prompt and provider transform paths sharing the same request-local projection memory.
  • Provider Aggregate Guard: 1 transform changed from aggregate-disabled to finite aggregate budget. This is the behavior change most likely to affect model-facing session history during active tool loops.

Root-cause cluster
Relationship: fixed_by_candidate
Canonical: #95219
Summary: This PR is a candidate fix for the canonical historical tool-result prompt-cache instability issue; related open PRs overlap, and none has merged into current main.

Members:

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🌊 off-meta tidepool
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • [P2] Choose this PR or one related candidate as the canonical fix for the linked prompt-cache issue.
  • Update the PR body so config/docs and validation claims match the actual branch diff before marking ready.

Risk before merge

  • [P1] The PR intentionally changes model-facing session-history projection: older tool-result bytes can be frozen for cache stability while newer eligible text may be cleared when the aggregate budget cannot otherwise be met.
  • [P1] Several open PRs target the same canonical issue, so maintainers should choose exactly one live prompt-projection policy before merging overlapping approaches.
  • [P1] The PR body still describes config/docs ceiling changes, but the live diff only touches agent runtime and tests; the body should distinguish already-on-main config/docs context from branch-local changes before ready-for-review merge decisions.

Maintainer options:

  1. Choose This Projection Policy
    Maintainers can accept this branch if they want request-local historical tool-result projections to stay byte-stable even when aggregate pressure shifts to newer eligible tool-result text.
  2. Select A Different Candidate
    Maintainers can pause or close this branch if they prefer the live-quantum or marker-skipping policy from another open candidate after reviewing the same canonical issue.
  3. Align Evidence Before Ready
    Before leaving draft, update the PR body so validation and config/docs claims match the actual branch diff and current-main context.

Next step before merge

  • [P2] Protected maintainer draft changes model-facing session history and overlaps open candidates, so maintainer policy/review is the next action rather than repair automation.

Security
Cleared: No concrete security or supply-chain concern is visible; the diff touches agent prompt projection logic and colocated tests only.

Review details

Best possible solution:

Land one reviewed stable live prompt-projection policy for #95219 with focused regression coverage and accurate PR evidence, then close the issue after the chosen fix reaches main.

Do we have a high-confidence way to reproduce the issue?

Yes, at source level: current main rebuilds aggregate tool-result projections from the evolving message set and spends reductions on older entries first. I did not run a live provider cache trace in this read-only review.

Is this the best way to solve the issue?

Unclear as the final landing path: the PR touches the right owner boundary with per-attempt projection state, but maintainers need to choose this policy over the related open candidate fixes and accept the retained-context tradeoff.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against adb9abe7215e.

Label changes

Label justifications:

  • P2: This is a normal-priority agent prompt-cache/session-state bugfix with real long-session latency and cost impact but limited blast radius.
  • merge-risk: 🚨 session-state: The PR changes how historical tool-result content is projected into model-facing session history across adjacent turns.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🌊 off-meta tidepool and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Not applicable: The external-contributor proof gate is not applicable because this is a MEMBER-authored maintainer-labeled draft PR; the PR body lists focused tests as supporting validation.
Evidence reviewed

PR surface:

Source +217, Tests +119. Total +336 across 4 files.

View PR surface stats
Area Files Added Removed Net
Source 2 243 26 +217
Tests 2 124 5 +119
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 4 367 31 +336

Acceptance criteria:

  • [P1] node scripts/run-vitest.mjs src/agents/embedded-agent-runner/tool-result-truncation.test.ts src/agents/embedded-agent-runner/run/attempt.spawn-workspace.context-engine.test.ts.
  • [P1] node scripts/check-database-first-legacy-stores.mjs.
  • [P1] git diff --check.

What I checked:

Likely related people:

  • vincentkoc: Local blame attributes the current main truncation helper and prompt caller lines to a recent Vincent Koc commit, and this PR is authored from the same area. (role: recent current-main carrier and likely follow-up owner; confidence: medium; commits: 328a44695fbb, 708c1b31e097; files: src/agents/embedded-agent-runner/tool-result-truncation.ts, src/agents/embedded-agent-runner/run/attempt.ts)
  • ooiuuii: Merged pull request Bound aggregate persisted tool results #87639 shows ooiuuii authored the aggregate prompt-boundary tool-result truncation behavior this PR refines. (role: introduced adjacent behavior; confidence: high; commits: 4f54861333fb, f49a3e4c266c; files: src/agents/embedded-agent-runner/tool-result-truncation.ts, src/agents/embedded-agent-runner/run/attempt.ts)
  • steipete: Live metadata for Bound aggregate persisted tool results #87639 shows steipete merged the adjacent aggregate prompt-boundary change, and the pre-merge history includes adjacent work on the same files. (role: adjacent merger and co-author; confidence: high; commits: 4f54861333fb, f49a3e4c266c; files: src/agents/embedded-agent-runner/tool-result-truncation.ts, src/agents/embedded-agent-runner/run/attempt.ts)
  • L-Trump: The open related pull request fix(cache): stable tool result aggregate truncation #93186 targets the same stable aggregate tool-result truncation problem with overlapping files and runtime cache-trace proof. (role: adjacent candidate-fix author; confidence: medium; commits: b2278f63ee87; files: src/agents/embedded-agent-runner/tool-result-truncation.ts, src/agents/embedded-agent-runner/tool-result-truncation.test.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. P2 Normal backlog priority with limited blast radius. merge-risk: 🚨 session-state 🚨 May lose, corrupt, stale, or mis-associate session, agent, or context state. labels Jun 22, 2026
@vincentkoc
vincentkoc merged commit 3053cbc into main Jun 22, 2026
104 of 110 checks passed
@vincentkoc
vincentkoc deleted the fix/stable-tool-result-prompt-cache branch June 22, 2026 01:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent runtime and tooling maintainer Maintainer-authored PR merge-risk: 🚨 session-state 🚨 May lose, corrupt, stale, or mis-associate session, agent, or context state. P2 Normal backlog priority with limited blast radius. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. size: M status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant