Skip to content

[WIP] Add support for video data - #478

Draft
chanwutk wants to merge 1 commit into
ucbepic:mainfrom
chanwutk:main
Draft

[WIP] Add support for video data#478
chanwutk wants to merge 1 commit into
ucbepic:mainfrom
chanwutk:main

Conversation

@chanwutk

@chanwutk chanwutk commented Mar 6, 2026

Copy link
Copy Markdown
Contributor

Data type syntax

[ ...
  {"video_content": {"__docetl_type__": "Video", "source": "xxxx.mp4"}}
... ]

Check list:

  • other video-related data types: VideoView (video with timestamp range), EntityView (crop-over-time of entity in the video).
  • Other operators support for video data.

Re-implement video content handling for MapOperation prompts
using a custom Jinja2 `as_video` filter approach:

- Add `_as_video_filter` in validation.py that emits unique
  delimited placeholders for Video-typed template values
- Add `rendered_prompt_to_content_parts` to convert rendered
  prompt strings containing video placeholders into LiteLLM
  multimodal content parts (type: file)
- Register the `as_video` filter in both the template
  validation environment and the strict_render environment
- Validate that a Gemini model is configured when video
  content is detected in prompts

Made-with: Cursor
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant