Skip to content

Dashboard: Spotify Web Playback widget (browser becomes a Spotify Connect device) #15182

@teknium1

Description

@teknium1

Follow-up to the Spotify integration rollout (#15121, #15130, #15135, #15154, #15174, #15180).

The goal

A dashboard plugin that makes the browser tab itself a Spotify Connect device. Audio plays out of the browser. The agent can control it directly via the existing spotify_devices and spotify_playback tools — no new backend code, it just shows up as a device the agent can target.

Why this tier (vs. a remote-control widget)

A simpler "remote control" widget would just poll get_currently_playing and show album art + play/pause buttons that call the existing tools. Useful, but it only controls a Spotify client you're already running elsewhere (phone, desktop app, Sonos).

The Web Playback SDK approach is strictly more capable: the browser IS the client. You don't need Spotify open anywhere else. The agent can say 'play X' from a fresh cron session and audio comes out of the dashboard tab that's already open. This is the natural home for a Hermes user who lives in the dashboard anyway.

Requirements

  • Premium only. Spotify's Web Playback SDK refuses to initialize on Free accounts. Widget should detect and show a clear message, not break silently.
  • Token handoff from auth.json to browser. The dashboard is a Hermes plugin with server-side access to ~/.hermes/auth.json. It needs to hand the Spotify access token to the browser for SDK init, and handle refresh (the SDK calls a user-provided callback when the token expires — that callback should hit a dashboard endpoint that returns a fresh token, refreshing via our existing _refresh_spotify_oauth_state if needed).
  • Device registration. The SDK emits a ready event with a device_id. Widget should expose that device in the UI ("This tab is registered as 'Hermes Dashboard' — agent can target it").
  • Control surface. Standard transport bar (play/pause/skip/seek, volume slider, current track + album art). Progress bar ticks locally between SDK state updates.
  • Agent-triggered playback. When the agent calls spotify_playback play with device_id = this tab's id, the SDK handles it. Nothing special from our side — it just works because the browser is a real Connect device.
  • Graceful degradation. If the tab loses focus / network / SDK connection, reconnect automatically. Don't leave zombie devices in the user's Spotify Connect list.

Reference

  • Web Playback SDK overview
  • SDK quick start
  • Existing Hermes dashboard plugin pattern: plugins/example-dashboard/, plugins/strike-freedom-cockpit/
  • Existing Spotify client module: plugins/spotify/client.py (has refresh logic we should reuse)
  • Existing auth: hermes_cli/auth.py _refresh_spotify_oauth_state and get_provider_auth_state("spotify")

Scope estimate

  • Dashboard plugin: ~400 LOC (SDK bootstrap, token refresh endpoint, transport UI, device state sync)
  • No changes to the core Spotify integration
  • Opt-in — only users who load the dashboard plugin pay for it

Non-goals

  • Playlist/library browsing UI. The agent handles that through tools. Widget stays focused on "what's playing + transport controls."
  • Offline playback. SDK doesn't support it.
  • Replacing the existing remote-control tools. The transport buttons in the widget should call the same spotify_playback tools so an external agent can drive the UI too.

Acceptance

  • User opens dashboard → sees Spotify widget showing login state
  • Clicks login → PKCE flow completes → tab shows 'Hermes Dashboard' as active device
  • Agent 'play miles davis' → music plays in the tab
  • Skip/pause buttons in the widget work and the agent sees the state change via get_currently_playing
  • Free users see a clear 'Spotify Premium required for in-browser playback' message

Metadata

Metadata

Assignees

No one assigned

    Labels

    P3Low — cosmetic, nice to havecomp/gatewayGateway runner, session dispatch, deliverycomp/pluginsPlugin system and bundled pluginstype/featureNew feature or request

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions