Skip to content

[TESTING][MCP-PROTOCOL]: Add Python SDK v2 live-stack E2E tests for MCP 2026-07-28 #5761

Description

@lucarlig

✅ Test Summary

After #5760 lands, add black-box end-to-end coverage for MCP 2026-07-28 using the official Python MCP SDK v2 against the ported live stack's public Nginx endpoint.

This is a concrete implementation task under #5557. It complements the official conformance runner with real Python SDK interoperability; it does not replace conformance.

Dependency

Remove the blocked label when #5760 is complete and the in-repository split-stack targets are available.

SDK and fixture requirements

  • Use the official Model Context Protocol Python SDK v2.
  • Pin an exact v2 release in the test environment. The current release is v2.0.0b2; use stable v2 if it has shipped when implementation starts.
  • Use the real mcp.Client for positive flows, not a mocked client or a hand-written JSON-RPC substitute.
  • Add a Python SDK v2 MCPServer fixture to the existing live-stack profile where a modern/dual-era upstream is required. Register it through the same control-plane APIs and wait for publisher visibility before dataplane tests.
  • Raw HTTP may be used only for malformed-message and header-mismatch deny paths that the SDK intentionally cannot emit.
  • Keep the SDK v2 dependency isolated to the test/live-stack environment unless [EPIC][MCP-PROTOCOL]: Migrate to Python MCP SDK 2.0.0 #5559 has already migrated the repository-wide dependency.

Coverage matrix

Run the same assertions through both public routes:

Lane Public route Required backend
Control plane /mcp controlplane
Dataplane /servers/{virtual_host_id}/mcp dataplane (never silent fallback)

Exercise these Python SDK v2 connection modes:

  • mode="auto": server/discover selects 2026-07-28 on the modern fixture.
  • mode="2026-07-28": the client uses a pinned modern connection without a negotiation round trip.
  • mode="2026-07-28", prior_discover=...: reconnect reuses the saved discovery result while preserving capabilities and server identity.
  • mode="legacy": the dual-era fixture remains usable through the initialization-based path.
  • mode="auto" against a legacy-only fixture: the SDK falls back from server/discover to initialize and reports the negotiated legacy version.

Protocol scenarios

Modern stateless core

  • server/discover returns the supported protocol versions, server info, and capabilities through both lanes.
  • A modern connection performs useful requests without initialize, notifications/initialized, or Mcp-Session-Id.
  • Independent requests and reconnects can be served without sticky sessions.
  • tools/list and tools/call work through the SDK and preserve structured content.
  • resources/list, resources/read, prompts/list, and prompts/get work when advertised by the fixture.
  • Pagination cursors round-trip through the gateway without mutation.
  • Tool input/output using full JSON Schema 2020-12, including composition/$defs and non-object structuredContent, round-trips unchanged.
  • Resource-not-found uses JSON-RPC -32602.

HTTP routing and validation

  • Modern SDK requests carry MCP-Protocol-Version: 2026-07-28 plus the required Mcp-Method and, where applicable, Mcp-Name headers.
  • Nginx and both gateway lanes preserve these headers end to end.
  • Missing or body-mismatched Mcp-Method/Mcp-Name is rejected with HTTP 400.
  • Dataplane assertions fail if X-CF-Integration-Backend reports controlplane-fallback.
  • Client info and supported trace-context metadata in _meta survive proxying without leaking credentials.

Modern response behavior

  • ttlMs and cacheScope metadata survive list/resource responses, and the Python client's cache behavior is exercised for one deterministic fixture.
  • At least one multi-round-trip InputRequiredResult flow is completed by the Python SDK v2 client when the corresponding ContextForge capability is available.
  • Known protocol gaps discovered by these tests are recorded as linked implementation issues; do not make permanent expected failures the success condition for this task.

Security and failure paths

  • Unauthenticated requests are rejected.
  • A token scoped to the wrong team cannot access the test virtual server.
  • Unsupported/invalid protocol versions fail closed.
  • Authorization tokens do not appear in URLs, subprocess arguments, pytest output, Nginx logs, or retained artifacts.

Live-stack integration

  • Add a focused Make target such as make test-mcp-2026-07-28-e2e.
  • Include the suite in the ported live-protocol and live-all groups.
  • Reuse the current live-stack lifecycle, fixture registration, publisher wait, backend identity, and cleanup helpers from [FEATURE][CF-DATAPLANE]: Port the split-plane live-stack harness into ContextForge #5760.
  • Keep test setup deterministic and clean up created gateways, virtual servers, users/teams, and containers when the suite fails.
  • Run the suite with both the control-plane and dataplane topologies and report failures by lane.

Acceptance criteria

  • A clean checkout can run one documented command that starts the required live stack, executes the Python SDK v2 suite, and tears the stack down.
  • All matrix rows pass against the pinned SDK version.
  • The dataplane lane proves requests were served by the published dataplane image rather than the control-plane fallback.
  • Existing 2025-11-25 live tests and official conformance targets continue to pass.
  • The new suite is part of the repository's live protocol test workflow and produces actionable per-lane failures.

References

Metadata

Metadata

Assignees

Labels

blockedBlocked by some other predecessor issue or PR. See notes.mcp-2026-07-28Issues related to compliance with MCP 2026-07-28mcp-protocolAlignment with MCP protocol or specificationpythonPython / backend development (FastAPI)testingTesting (unit, e2e, manual, automated, etc)triageIssues / Features awaiting triage

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions