Problem

Emoji overlay positioning for Arabic (and Hebrew) RTL text is incorrect in the production captions.ts pipeline. The emoji overlaps with or is misplaced relative to the Arabic text because captions.ts uses getCharXPositions() for all scripts, including RTL.

getCharXPositions() computes X positions in logical (LTR) character order, but libass renders Arabic text RTL with BiDi reordering via HarfBuzz. The logical character positions don't map to the visual X positions on screen.

Visual evidence

In the Rendi E2E test for "مرحبا 💪 بالعالم", the emoji (💪) overlaps with the beginning of "بالعالم" instead of sitting cleanly in the gap between the two Arabic words.

Root cause

1. getCharXPositions() computes character advances using opentype.js charToGlyph() (direct cmap lookup, no BiDi/GSUB shaping). For Arabic, this returns isolated-form glyph widths which overestimate by ~28-35% vs what HarfBuzz renders (contextual forms + ligatures).

2. Even with the measureRenderedWidth() fallback (which measures actual rendered width via ffmpeg pixel scanning and distributes advances evenly), the positions are in logical order — the first character in the string maps to the leftmost position. But libass renders Arabic right-to-left, so the first Arabic character is actually on the right side of the line.

3. BiDi reordering means emoji (which are LTR neutral characters surrounded by RTL text) can end up at unpredictable visual positions depending on the Unicode BiDi algorithm implementation in HarfBuzz/libass.

Solution already implemented (in visual test only)

The emoji-position-test.ts visual diagnostic script already has the correct approach:

if (hasRTL) {
  const gaps = measureEmojiGapPositions(
    taggedText, fontName, fontSize, fontDir,
    videoWidth, videoHeight, alignment,
    marginL, marginR, marginV,
    emojiSize, emojiCount,
  );
  // Use gap.x for each emoji overlay position
}

measureEmojiGapPositions() (in text-measure.ts) works by:

1. Rendering the full subtitle line with ffmpeg (subtitles filter on a black frame)
2. Scanning the output frame column-by-column to find gaps (runs of empty columns wider than emojiSize * 0.4)
3. Returning the gap center X for each emoji

This approach works regardless of BiDi reordering because it operates on the actual rendered pixels, not logical character positions.

TODO

• Integrate measureEmojiGapPositions() into captions.ts for RTL scripts
• Add RTL detection: detectScriptsInText(text) already returns script sets — check for "arabic" or "hebrew"
• For RTL text with emoji, call measureEmojiGapPositions() instead of getCharXPositions() to compute overlay X positions
• The gap detection requires local font files and ffmpeg — ensure ensureLocalFonts() is called before measurement (it already is in the current pipeline)
• Test with: emoji at beginning of Arabic text, emoji at end, emoji in middle, mixed English+Arabic+emoji
• Note: measureEmojiGapPositions() adds ~200ms per subtitle line (spawns ffmpeg) — acceptable for caption rendering but worth noting

Affected files

• src/react/renderers/captions.ts — needs RTL detection + gap-based positioning
• src/react/renderers/text-measure.ts — measureEmojiGapPositions() already implemented
• src/react/renderers/emoji-position-test.ts — already uses the correct approach for RTL (reference implementation)

Related

• Branch: feature/caption-fonts
• All 43 visual test cases pass (including Arabic) when using measureEmojiGapPositions()
• The Rendi E2E test for Arabic+emoji passes (Rendi renders it) but the emoji position is visually wrong