feat: telemetry for memory manager

[opieter-aws]

Description

Motivation

Observability is a core feature of the SDK, but the memory subsystem was the one major agent subsystem with no OpenTelemetry instrumentation — it emitted only logs. Every other component (agent invocation, event-loop cycle, tool execution, model invocation) flows through the Tracer for spans, so memory operations were invisible in traces: you couldn't see how often memory was searched, how slow stores were, how many entries were retrieved/injected/extracted, or why a background extraction failed. Notably, the memory extractor's model call was the only model invocation in either SDK not wrapped in a model-invoke span, leaving extraction cost and latency completely dark.

This change instruments the memory subsystem in both the Python and TypeScript SDKs so it matches the rest of the SDK: spans for search, add, context injection, and background extraction, plus wrapping the extraction model call in the existing model-invoke span. Spans nest correctly under the agent/cycle/tool traces.

This PR is tracing/spans only. An earlier revision also added strands.memory.* metrics; those were dropped after review feedback that the metric set was noisy and the store_name dimension risked unbounded cardinality. Metrics can follow in a separate PR once there is a concrete dashboard need — most of what they offered is already derivable from span data.

Public API Changes

No existing signature on Agent or MemoryManager changed. The new surface is the span schema exporters receive, and the Tracer methods that produce it.

Span schema — what shows up in a tracing backend. Span names use a memory.* namespace alongside the standard gen_ai.operation.name; the two SDKs are kept at exact parity on names, attributes, and events.

Span	Emitted when	Parenting	Key attributes	Events
memory.search	a store query runs (tool or injection)	active agent span	memory.store.names, memory.store.count, memory.max_search_results, memory.result.count, memory.store.failure_count	memory.query, memory.results
memory.add	a write runs	active agent span, or root for the fire-and-forget add-tool path	memory.store.names, memory.store.count, memory.store.failure_count	memory.content
memory.inject	context injection runs before a model call	active agent span	memory.max_entries, memory.injected, memory.entry.count, memory.inject.format_error	—
memory.extract	background extraction runs	root, linked to scheduling agent span	memory.store.name, memory.message.count, memory.message.filtered_count, memory.extractor, memory.entry.count, memory.parent.trace_id, memory.parent.span_id	—

New Tracer methods — a start/end pair per operation, following the existing start*Span/end*Span convention. TypeScript (strands-ts/src/telemetry/tracer.ts) is options-object based and returns Span | null:

startMemorySearchSpan(options: StartMemorySearchSpanOptions): Span | null
endMemorySearchSpan(span: Span | null, options?: EndMemorySearchSpanOptions): void

startMemoryAddSpan(options: StartMemoryAddSpanOptions): Span | null        // options.forceRoot for detached writes
endMemoryAddSpan(span: Span | null, options?: EndMemoryAddSpanOptions): void

startMemoryInjectSpan(options?: StartMemoryInjectSpanOptions): Span | null
endMemoryInjectSpan(span: Span | null, options: EndMemoryInjectSpanOptions): void

startMemoryExtractSpan(options: StartMemoryExtractSpanOptions): Span | null  // options.agentSpanContext attached as a link
endMemoryExtractSpan(span: Span | null, options?: EndMemoryExtractSpanOptions): void

currentSpanContext(): SpanContext | undefined   // @internal — captures the live agent span to link extraction back to it

The matching StartMemory*SpanOptions / EndMemory*SpanOptions interfaces are exported from strands-ts/src/telemetry/types.ts. Python (strands-py/src/strands/telemetry/tracer.py) mirrors these as keyword-argument methods:

start_memory_search_span(query, store_names, max_search_results=None, *, custom_trace_attributes=None, **kwargs) -> Span
end_memory_search_span(span, entries=None, store_failure_count=0, error=None) -> None

start_memory_add_span(content, store_names, *, force_root=False, custom_trace_attributes=None, **kwargs) -> Span
end_memory_add_span(span, store_failure_count=0, error=None) -> None

start_memory_inject_span(max_entries=None, *, custom_trace_attributes=None, **kwargs) -> Span
end_memory_inject_span(span, injected, entry_count=0, format_error=False) -> None

start_memory_extract_span(store_name, message_count, filtered_count=0, extractor=None, agent_span_context=None, **kwargs) -> Span
end_memory_extract_span(span, entry_count=None, error=None) -> None

Design notes

• Detached extraction is a root span, not a child. Automatic extraction runs in a background task (Python asyncio, TS promise chain) that typically completes after the agent span has already ended. Parenting to it would orphan the span, so extraction starts its own root trace and carries an OpenTelemetry link back to the agent span — captured synchronously while that span is still live. This preserves causal association without an orphaned parent-child relationship.

• Telemetry never breaks memory. Span failures are isolated so a misbehaving tracer provider can never turn a successful search/add/extraction into a failure. The detached extraction ends its span best-effort after the save resolves, so a tracing error can't roll back a successful write or trip the store's backoff logic.

• Injection fails open and never leaks a span. The injection path ends its span on every branch — no query, no results, a throwing format callback (recorded as a flag, not an error, since the agent did not fail), and an unexpected search error (span ended, then rethrown).

• Content on span events. Query and stored/retrieved content are recorded as span events, consistent with how model and tool spans record their inputs. Memory payloads may contain PII; this is documented on the span methods so operators can suppress span content at the exporter when needed.

• Per-SDK tracer-acquisition idiom. Python uses the existing global get_tracer(). TypeScript's Tracer is per-instance, so MemoryManager constructs one and shares it with the extraction coordinator; parenting still flows through OTel context, so this does not fragment traces.

Related Issues

Documentation PR

Type of Change

New feature

Testing

How have you tested the change? Verify that the changes do not break functionality or introduce new warnings.

• I ran hatch run prepare

Unit tests in both SDKs cover the span lifecycle for each operation (success, error, and early-return paths), the detached-extraction root-span-with-link behavior, telemetry-failure isolation, fail-open injection, the model-invoke span on the extractor call, and (TS) span-context validity/recording checks. Verified end to end with in-memory span exporters in both SDKs: injection nests a child search span, a tool-invoked search nests under the tool span, and extraction appears as its own root trace linked back to the agent span.

Checklist

• I have read the CONTRIBUTING document
• I have reviewed and understand every line of code in this PR, including any generated by AI tools, and I can explain why it works
• My change is focused and reasonably small; I have split unrelated work into separate PRs
• I have added any necessary tests that prove my fix is effective or my feature works
• I have updated the documentation accordingly
• I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
• My changes generate no new warnings
• Any dependent changes have been merged and published

---

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[github-actions]

Assessment: Comment

Well-structured feature that fills a real observability gap, and the PR description does an excellent job explaining the why (detached-extraction-as-root-span-with-link, telemetry-as-best-effort, content-as-span-events). Tests are thorough — success/error/early-return paths, telemetry-failure isolation, and the detached link behavior are all covered, and they pass locally. My feedback is mostly about consistency, not correctness.

Review themes

• Telemetry isolation is uneven: the "telemetry never breaks memory" guarantee is enforced for background extraction but not for the foreground search/add/inject paths. Worth either extending the guard or documenting the intent. (inline)
• Minor metric/type consistency: one histogram recorded without an attribute dimension, and one list[Any] that could be list[MemoryEntry]. (inline)
• API review: this adds a meaningful public surface (5 start_*/end_* Tracer methods + 16 MetricsClient instruments). The description already documents signatures and the strands.memory.* namespace well, which is most of the bar-raising prep — but per team/API_BAR_RAISING.md this looks like a "moderate" change that should carry the api/needs-review label and a quick reviewer sign-off before merge.

Nicely done overall — the design reasoning is unusually clear and the extraction-coordinator failure handling is careful.

[poshinchen]

I think the otel metrics can be skipped as of now? they might be noisy

[opieter-aws]

@poshinchen will scope the PR to tracing/spans only for now, and expand to TS

[github-actions]

Re-review after rescope — Assessment: Comment (close to Approve)

Picking up the changes since my first pass: metrics were dropped per @poshinchen's noise concern (39715270) and a full TypeScript port was added (f798459). Re-ran both suites locally — Python 122 passed, TS 154 passed, no type errors.

What I verified this round

• Metrics removal is clean: no MetricsClient/metrics_client references remain under src/strands/memory, and test_telemetry.py was reworked to spans-only. Two of my earlier inline comments (inject-metric attributes, foreground telemetry isolation) are now moot and I've resolved them.
• TS port mirrors the Python design faithfully: detached memory.extract as a trace root + agent-span link captured synchronously at schedule time, best-effort span ending decoupled from save outcome (separate try/catch), mark rollback on failure. The extraction-model call is wrapped in the model-invoke span on both sides.
• Per-language tracer conventions are respected: the new TS span methods are try/catch-guarded like the rest of the TS tracer; the Python ones are unguarded like the existing start_model_invoke_span. Consistent within each language — no cross-language defect.
• One carryover nit: TS types entries as MemoryEntry[], but Python still uses list[Any]. Worth aligning Python (left on the existing thread).

The scope reduction tightened this up nicely and the TS implementation is a careful 1:1 of the Python semantics. The only remaining item is the cosmetic list[Any] → list[MemoryEntry] alignment; the API-review-label suggestion from my first summary still applies to the public Tracer surface in both languages.

[github-actions]

Re-review after ded3a77c — Assessment: Approve (pending API-label process)

The latest commit cleanly resolves the only outstanding code nit: entries is now list[MemoryEntry] | None (imported under TYPE_CHECKING, no circular-import risk), bringing Python in line with the TS MemoryEntry[]. Verified locally — mypy clean, 95 tracer tests pass, and the new inject-span test closes the codecov partial line.

No remaining code-level concerns. The one open item is process, not code: per team/API_BAR_RAISING.md this still looks like a "moderate" public-API change (the start_*/end_* memory Tracer methods in both Python and TS) that would benefit from the api/needs-review label and a quick reviewer sign-off before merge.

Nicely iterated — the design reasoning and failure handling have been solid throughout.

[github-actions]

Re-review after 8ca32d8 — Assessment: Approve (pending API-label process)

This follow-up makes two small, well-scoped improvements, both verified locally (123 tracer + memory-telemetry tests pass, mypy clean, Python/TS kept in parity):

What changed

• Langfuse span-attribute promotion — the memory.query / memory.results / memory.content events now pass to_span_attributes=self.is_langfuse. This isn't a new pattern; it mirrors exactly how the existing model/tool/agent spans already promote event content so backends that don't render span events (Langfuse) still surface it. Good consistency win.
• Parent ids on the detached extract span — memory.parent.trace_id / memory.parent.span_id are now recorded as plain attributes alongside the existing OTel Link. The inline comment explains the rationale clearly (the detached root has no OTel parent, and some backends don't render span links), the is_valid guard correctly gates the link and both ids together, and it uses the proper format_trace_id/format_span_id helpers. Both valid and invalid-context paths are tested in Python and TS.

No remaining code-level concerns. The only open item is still process, not code: per team/API_BAR_RAISING.md, the public start_*/end_* memory Tracer methods (Python + TS) look like a "moderate" public-API addition that would benefit from the api/needs-review label and a quick reviewer sign-off before merge.

Solid, focused iteration.