feat(webrtc): allow switching to an audio-only rendition in a video session - #2212
Open
naanlizard wants to merge 1 commit into
Open
feat(webrtc): allow switching to an audio-only rendition in a video session#2212naanlizard wants to merge 1 commit into
naanlizard wants to merge 1 commit into
Conversation
…ession
RtcMasterPlaylist buckets renditions by (video_codec_id, audio_codec_id),
so an audio-only rendition (video codec None) lands in its own playlist that
a session which negotiated video never sees. A video session therefore cannot
switch to an audio-only rendition on the same PeerConnection.
When the WebRtcAudioOnlyFallback playlist option is enabled, cross-add each
audio-only rendition into every video playlist that shares its audio codec, and
back-fill video playlists created after the audio-only rendition is seen. The
triggering video rendition is added before any cross-added audio-only rendition,
so it remains the playlist's default rendition (GetFirstRendition). The option
is off by default, so existing configurations are unaffected.
Within a video playlist, automatic bitrate ABR never moves a session onto, or off
of, an audio-only rendition: GetNext{Higher,Lower}BitrateRendition skip audio-only
candidates so congestion never drops a video session to audio-only, and they pin a
session whose current rendition is audio-only so it is not auto-resumed to video.
Audio-only is therefore reachable only by an explicit change_rendition request and
the session stays there until the client switches back. Both behaviours are gated
on the playlist being a video playlist, so an audio-only playlist still performs
normal bitrate ABR among its own renditions.
Audio-only renditions are advertised in the playlist notification so a client
can request them by name. rtc_session already supports switching to a video-less
rendition without renegotiation.
Add unit tests for RtcMasterPlaylist/RtcPlaylist covering cross-add and back-fill
ordering, the video-rendition-stays-default invariant, ABR exclusion in a video
playlist (including an audio-only rendition placed between two video bitrates and
the pinning of an explicitly-selected audio-only rendition against auto-ABR), ABR
among renditions in an audio-only playlist, codec-mismatch and multiple-video
-playlist cases, video-only renditions with no audio track, and the option being
disabled by default. Document the option in the WebRTC publishing guide.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
naanlizard
force-pushed
the
feature/webrtc-audio-only-rendition
branch
from
June 19, 2026 21:06
0b07aec to
368fb28
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
This PR enables a WebRTC video session to explicitly switch to an audio-only rendition (same audio codec) without renegotiation by making audio-only renditions visible inside matching video playlists when a new config option is enabled.
Changes:
- Add
WebRtcAudioOnlyFallbackplaylist option (default off) and plumb it from config →info::Playlist→RtcStream→RtcMasterPlaylist. - Update
RtcMasterPlaylistto cross-add/back-fill audio-only renditions into matching video playlists, and updateRtcPlaylistABR to never auto-select (or auto-exit) audio-only within video playlists. - Add gtest coverage for cross-add/back-fill ordering and ABR semantics; document the new option.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| src/projects/publishers/webrtc/webrtc_test.cpp | Adds gtests validating cross-add/back-fill behavior and ABR exclusion/pinning semantics. |
| src/projects/publishers/webrtc/rtc_stream.cpp | Plumbs the new playlist option into RtcMasterPlaylist creation. |
| src/projects/publishers/webrtc/rtc_playlist.h | Implements audio-only cross-add/back-fill and ABR guardrails for video playlists. |
| src/projects/config/items/virtual_hosts/applications/output_profiles/playlist/playlist.h | Copies the parsed option into info::Playlist. |
| src/projects/config/items/virtual_hosts/applications/output_profiles/playlist/options.h | Adds the WebRtcAudioOnlyFallback config option and default. |
| src/projects/base/info/playlist.h | Stores, prints, compares, and exposes the new option on info::Playlist. |
| docs/streaming/webrtc-publishing.md | Documents the new option and provides an XML example. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
|
||
| If `<Options>/<WebRtcAutoAbr>` is set to true, OvenMediaEngine will measure the bandwidth of the player session and automatically switch to the appropriate rendition. | ||
|
|
||
| If `<Options>/<WebRtcAudioOnlyFallback>` is set to true, OvenMediaEngine cross-adds any audio-only rendition of the playlist (a `<Rendition>` with no `<Video>`) into the video renditions that share its audio codec. A WebRTC player can then switch to the audio-only rendition on the same PeerConnection without renegotiation — for example to keep audio while dropping video bandwidth when the player is in the background. The option is off by default. Automatic ABR never switches to the audio-only rendition; it is reachable only by an explicit rendition change, and a session that has switched to it stays there until the player switches back. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
One more PR for now, again full disclosure all coded by claude, this is a much smaller change though.
The very short version is that this lets us downgrade background-tab WebRTC streams to audio only, saving a bunch of bandwidth
Problem
RtcMasterPlaylistbuckets renditions by(video_codec_id, audio_codec_id), so anaudio-only rendition (video codec
None) lands in its own(None, audio)playlist thata session which negotiated video never sees. A WebRTC video session therefore cannot
switch to an audio-only rendition on the same
PeerConnection— which is useful forkeeping audio while dropping all video traffic without renegotiating, for example when a
player is backgrounded or hidden.
Change
A new playlist option
WebRtcAudioOnlyFallback(off by default) enables the behaviour.When it is set:
audio codec, and back-filled into video playlists created after the audio-only
rendition is first seen (so config declaration order does not matter).
it stays the playlist default (
GetFirstRendition) — a session always starts on video.Because the option defaults to off, existing configurations are unaffected.
ABR semantics
Within a video playlist, automatic bitrate ABR never moves a session onto, or off of, the
audio-only rendition:
GetNextHigherBitrateRendition/GetNextLowerBitrateRenditionskip audio-onlycandidates, so congestion never drops a video session down to audio-only.
nullptrin bothdirections), so a session that switched to audio-only is not auto-resumed to video even
if the client left auto-ABR enabled.
So the audio-only rendition is reachable only by an explicit
change_rendition, and thesession stays there until the client switches back. Both behaviours are gated on the
playlist being a video playlist, so an audio-only playlist still performs normal bitrate
ABR among its own renditions.
No SDP or renegotiation changes are needed:
rtc_sessionalready supports switching to avideo-less rendition on an established
PeerConnection, and the audio-only rendition isadvertised in the
playlistnotification so a client can request it by name.Config example
The video and audio-only renditions must share the same audio codec for the cross-add to
apply. The option is documented in
docs/streaming/webrtc-publishing.md.Testing
Adds gtest coverage in
src/projects/publishers/webrtc/webrtc_test.cppforRtcMasterPlaylist/RtcPlaylist: cross-add and back-fill ordering, thevideo-rendition-stays-default invariant, ABR exclusion of audio-only in a video playlist
(including an audio-only rendition whose bitrate sits between two video renditions, and the
pinning of an explicitly-selected audio-only rendition against auto-ABR), normal bitrate
ABR within an audio-only playlist, codec-mismatch and multiple-video-playlist cases,
video-only renditions with no audio track, and the option being disabled by default.