Skip to content

Fix terminal combining marks - #53176

Merged
ConradIrwin merged 5 commits into
zed-industries:mainfrom
MX48Live:fix-terminal-combining-marks
Apr 23, 2026
Merged

Fix terminal combining marks#53176
ConradIrwin merged 5 commits into
zed-industries:mainfrom
MX48Live:fix-terminal-combining-marks

Conversation

@MX48Live

@MX48Live MX48Live commented Apr 5, 2026

Copy link
Copy Markdown
Contributor

Self-Review Checklist:

  • I've reviewed my own diff for quality, security, and reliability
  • Unsafe blocks (if any) have justifying comments
  • The content is consistent with the UI/UX checklist
  • Tests cover the new/changed behavior
  • Performance impact has been considered and is acceptable

Closes #46942

Problem

The terminal's monospace grid forcing logic in LineLayoutCache breaks rendering of combining marks in complex scripts. Thai, Lao, Arabic, Hebrew, Myanmar, Khmer, and other scripts use combining characters (vowels, tone marks, diacritics) that should render on top of or below their base character — not in a separate cell.

The root cause is in apply_force_width within line_layout.rs. After HarfBuzz correctly shapes the text (positioning combining marks at the same x coordinate as their base character), the force-width loop blindly increments the cell position counter for every glyph — including zero-advance combining marks. This displaces them into the next cell, breaking the visual output.

For example, Thai สวัสดี renders as ส ว ั ส ด ี with vowels and tone marks scattered across separate cells.

#39526 previously fixed Latin combining characters (like NFD-normalized é) by handling them at the terminal cell level via append_zero_width_chars. That fix works when alacritty's cell model already identifies the characters as zero-width. However, many complex script combining marks still get mispositioned at the shaping layer — which is what this PR addresses.

Solution

Extracted apply_force_width_to_layout() in line_layout.rs that detects combining marks during the force-width pass by checking whether a glyph's shaped x position has advanced by at least half a cell width beyond the previous base character. Combining marks (which HarfBuzz places at the same x as their base) are kept at their shaped position relative to the forced base, rather than being pushed to the next cell.

This approach is script-agnostic — it works for all complex scripts (Thai, Arabic, Devanagari, Myanmar, Khmer, etc.) without hardcoding Unicode ranges or disabling the monospace grid. The fix lives at the text shaping layer in GPUI, so it benefits any consumer of LineLayoutCache that uses force_width.

The two duplicate force-width loops (in layout_line and layout_line_by_hash) are consolidated into the single shared helper.

Testing

Unit tests cover the following scenarios:

  • Latin text positions remain unchanged
  • A single combining mark stays at its base character's cell
  • A base character following combining marks gets the correct next cell
  • Multiple stacked combining marks all stay anchored to cell 0
  • Drifted base positions respect the existing 1px tolerance

Manual testing with the following scripts in the integrated terminal:

Script Test text What to look for
Thai สวัสดีครับ ภาษาไทย ก้อนหิน น้ำตก Vowel signs (ั ี) and tone marks (้) render above/below consonants, not in separate cells
Lao ສະບາຍດີ ພາສາລາວ Vowel signs stay attached to consonants
Arabic بِسْمِ اللَّهِ الرَّحْمَٰنِ Diacritics (harakat) render on their base letters
Hebrew בְּרֵאשִׁית בָּרָא אֱלֹהִים Vowel points (nikud) stay under/over their letters
Hindi नमस्ते हिन्दी भाषा Vowel matras render correctly on consonants
Bengali বাংলা ভাষা নমস্কার Combining marks stay attached
Tamil தமிழ் மொழி வணக்கம் Vowel signs render correctly
Myanmar မြန်မာဘာသာ မင်္ဂလာပါ Combining marks and medial consonants stay in place
Khmer ភាសាខ្មែរ សួស្តី Subscript consonants and vowel signs render correctly
Script to reproduce
echo "=== Thai ==="
echo "สวัสดีครับ ภาษาไทย ก้อนหิน น้ำตก"
echo ""
echo "=== Lao ==="
echo "ສະບາຍດີ ພາສາລາວ"
echo ""
echo "=== Arabic ==="
echo "بِسْمِ اللَّهِ الرَّحْمَٰنِ"
echo ""
echo "=== Hebrew ==="
echo "בְּרֵאשִׁית בָּרָא אֱלֹהִים"
echo ""
echo "=== Hindi (Devanagari) ==="
echo "नमस्ते हिन्दी भाषा"
echo ""
echo "=== Bengali ==="
echo "বাংলা ভাষা নমস্কার"
echo ""
echo "=== Tamil ==="
echo "தமிழ் மொழி வணக்கம்"
echo ""
echo "=== Myanmar ==="
echo "မြန်မာဘာသာ မင်္ဂလာပါ"
echo ""
echo "=== Khmer ==="
echo "ភាសាខ្មែរ សួស្តី"

Before

Screenshot 2569-04-05 at 10 43 00

After

Screenshot 2569-04-05 at 11 22 55

Tests passing

Screenshot 2569-04-05 at 11 20 22

Release Notes:

  • Fixed terminal rendering of combining marks in complex scripts (Thai, Arabic, Hebrew, Devanagari, Myanmar, Khmer, and others) where vowel signs and tone marks were incorrectly displaced to adjacent cells instead of rendering on their base characters.

MX48Live added 4 commits April 5, 2026 10:32
The force_width loop in LineLayoutCache incorrectly incremented the
cell position counter for every glyph, including combining marks
(zero-advance glyphs like Thai vowel signs and tone marks). This caused
combining marks to be displaced to the next cell instead of rendering
on top of their base character.

Detect combining marks by checking whether the shaped x position has
advanced by at least half a cell beyond the previous base character.
Unlike script-specific fixes, this handles all complex scripts (Thai,
Arabic, Devanagari, etc.) at the text shaping layer.

Closes zed-industries#46942
Tests verify that:
- Latin text positions are unchanged
- Combining marks stay at their base character's position
- Base characters after combining marks get the correct cell
- Multiple stacked combining marks all stay at cell 0
- Drifted base positions are corrected to cell boundaries
- Fix combining mark misalignment when base glyph is within 1px
  tolerance: track actual glyph position instead of grid slot
- Extract glyph_x_positions test helper to eliminate repetition
- Use f32::from() instead of direct .0 field access
- Remove redundant comments that restate assertions
@cla-bot cla-bot Bot added the cla-signed The user has signed the Contributor License Agreement label Apr 5, 2026
@zed-community-bot zed-community-bot Bot added the first contribution the author's first pull request to Zed. NOTE: the label application is automated via github actions label Apr 5, 2026
@zed-codeowner-coordinator
zed-codeowner-coordinator Bot requested review from a team, as-cii and dinocosta and removed request for a team April 5, 2026 04:29
@MX48Live

MX48Live commented Apr 7, 2026

Copy link
Copy Markdown
Contributor Author

Hi @as-cii @dinocosta 🙏

I've submitted this PR to fix the terminal combining marks issue (especially affecting Thai, Lao, Arabic, and other complex scripts where diacritics/vowels are rendered in wrong positions).

This is important for me ; w ; please consider

Thank you very much!

@ConradIrwin

Copy link
Copy Markdown
Member

Interesting. I had expected we'd want to fix this using something like the unicode segmentation crate to do this, but this seems to work effectively and avoids drift between the text renderer and our hypotheticals.

Willing to give it a go. Thanks for the examples.

@ConradIrwin
ConradIrwin enabled auto-merge (squash) April 23, 2026 02:39
@ConradIrwin
ConradIrwin merged commit debf4c9 into zed-industries:main Apr 23, 2026
31 checks passed
AzureZee added a commit to AzureZee/gpuix that referenced this pull request Apr 23, 2026
AzureZee added a commit to AzureZee/gpuix that referenced this pull request Apr 23, 2026
@staciax

staciax commented Apr 25, 2026

Copy link
Copy Markdown

tysm 😭

@kittizz

kittizz commented Apr 28, 2026

Copy link
Copy Markdown

ขอบคุณครับ เฝ้ารอ release นี้อย่างตั้งใจ

@ConradIrwin

Copy link
Copy Markdown
Member

ด้วยความยินดี!

kathbigra pushed a commit to kathbigra/zed that referenced this pull request May 10, 2026
Self-Review Checklist:

- [x] I've reviewed my own diff for quality, security, and reliability
- [x] Unsafe blocks (if any) have justifying comments
- [x] The content is consistent with the [UI/UX
checklist](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
- [x] Tests cover the new/changed behavior
- [x] Performance impact has been considered and is acceptable

Closes zed-industries#46942

## Problem

The terminal's monospace grid forcing logic in `LineLayoutCache` breaks
rendering of combining marks in complex scripts. Thai, Lao, Arabic,
Hebrew, Myanmar, Khmer, and other scripts use combining characters
(vowels, tone marks, diacritics) that should render on top of or below
their base character — not in a separate cell.

The root cause is in `apply_force_width` within `line_layout.rs`. After
HarfBuzz correctly shapes the text (positioning combining marks at the
same x coordinate as their base character), the force-width loop blindly
increments the cell position counter for **every** glyph — including
zero-advance combining marks. This displaces them into the next cell,
breaking the visual output.

For example, Thai `สวัสดี` renders as `ส ว ั ส ด ี` with vowels and tone
marks scattered across separate cells.

zed-industries#39526 previously fixed Latin combining characters (like NFD-normalized
é) by handling them at the terminal cell level via
`append_zero_width_chars`. That fix works when alacritty's cell model
already identifies the characters as zero-width. However, many complex
script combining marks still get mispositioned at the shaping layer —
which is what this PR addresses.

## Solution

Extracted `apply_force_width_to_layout()` in `line_layout.rs` that
detects combining marks during the force-width pass by checking whether
a glyph's shaped x position has advanced by at least half a cell width
beyond the previous base character. Combining marks (which HarfBuzz
places at the same x as their base) are kept at their shaped position
relative to the forced base, rather than being pushed to the next cell.

This approach is **script-agnostic** — it works for all complex scripts
(Thai, Arabic, Devanagari, Myanmar, Khmer, etc.) without hardcoding
Unicode ranges or disabling the monospace grid. The fix lives at the
text shaping layer in GPUI, so it benefits any consumer of
`LineLayoutCache` that uses `force_width`.

The two duplicate force-width loops (in `layout_line` and
`layout_line_by_hash`) are consolidated into the single shared helper.

## Testing

**Unit tests** cover the following scenarios:
- Latin text positions remain unchanged
- A single combining mark stays at its base character's cell
- A base character following combining marks gets the correct next cell
- Multiple stacked combining marks all stay anchored to cell 0
- Drifted base positions respect the existing 1px tolerance

**Manual testing** with the following scripts in the integrated
terminal:

| Script | Test text | What to look for |
|--------|-----------|-----------------|
| Thai | `สวัสดีครับ ภาษาไทย ก้อนหิน น้ำตก` | Vowel signs (ั ี) and tone
marks (้) render above/below consonants, not in separate cells |
| Lao | `ສະບາຍດີ ພາສາລາວ` | Vowel signs stay attached to consonants |
| Arabic | `بِسْمِ اللَّهِ الرَّحْمَٰنِ` | Diacritics (harakat) render
on their base letters |
| Hebrew | `בְּרֵאשִׁית בָּרָא אֱלֹהִים` | Vowel points (nikud) stay
under/over their letters |
| Hindi | `नमस्ते हिन्दी भाषा` | Vowel matras render correctly on
consonants |
| Bengali | `বাংলা ভাষা নমস্কার` | Combining marks stay attached |
| Tamil | `தமிழ் மொழி வணக்கம்` | Vowel signs render correctly |
| Myanmar | `မြန်မာဘာသာ မင်္ဂလာပါ` | Combining marks and medial
consonants stay in place |
| Khmer | `ភាសាខ្មែរ សួស្តី` | Subscript consonants and vowel signs
render correctly |

<details>
<summary>Script to reproduce</summary>

```sh
echo "=== Thai ==="
echo "สวัสดีครับ ภาษาไทย ก้อนหิน น้ำตก"
echo ""
echo "=== Lao ==="
echo "ສະບາຍດີ ພາສາລາວ"
echo ""
echo "=== Arabic ==="
echo "بِسْمِ اللَّهِ الرَّحْمَٰنِ"
echo ""
echo "=== Hebrew ==="
echo "בְּרֵאשִׁית בָּרָא אֱלֹהִים"
echo ""
echo "=== Hindi (Devanagari) ==="
echo "नमस्ते हिन्दी भाषा"
echo ""
echo "=== Bengali ==="
echo "বাংলা ভাষা নমস্কার"
echo ""
echo "=== Tamil ==="
echo "தமிழ் மொழி வணக்கம்"
echo ""
echo "=== Myanmar ==="
echo "မြန်မာဘာသာ မင်္ဂလာပါ"
echo ""
echo "=== Khmer ==="
echo "ភាសាខ្មែរ សួស្តី"
```

</details>

### Before

<img width="1144" height="740" alt="Screenshot 2569-04-05 at 10 43 00"
src="https://github.com/user-attachments/assets/bce2075a-9e19-40dd-81ea-0e29a451b781"
/>

### After

<img width="1144" height="747" alt="Screenshot 2569-04-05 at 11 22 55"
src="https://github.com/user-attachments/assets/e0613bf6-8452-4c65-b893-f9931a471b2e"
/>

### Tests passing

<img width="1144" height="740" alt="Screenshot 2569-04-05 at 11 20 22"
src="https://github.com/user-attachments/assets/9100bd77-4ea6-4f77-ae31-5129484552fd"
/>

Release Notes:

- Fixed terminal rendering of combining marks in complex scripts (Thai,
Arabic, Hebrew, Devanagari, Myanmar, Khmer, and others) where vowel
signs and tone marks were incorrectly displaced to adjacent cells
instead of rendering on their base characters.

---------

Co-authored-by: Conrad Irwin <conrad.irwin@gmail.com>
jonx pushed a commit to jonx/zed-aros that referenced this pull request Jul 17, 2026
Self-Review Checklist:

- [x] I've reviewed my own diff for quality, security, and reliability
- [x] Unsafe blocks (if any) have justifying comments
- [x] The content is consistent with the [UI/UX
checklist](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
- [x] Tests cover the new/changed behavior
- [x] Performance impact has been considered and is acceptable

Closes zed-industries#46942

## Problem

The terminal's monospace grid forcing logic in `LineLayoutCache` breaks
rendering of combining marks in complex scripts. Thai, Lao, Arabic,
Hebrew, Myanmar, Khmer, and other scripts use combining characters
(vowels, tone marks, diacritics) that should render on top of or below
their base character — not in a separate cell.

The root cause is in `apply_force_width` within `line_layout.rs`. After
HarfBuzz correctly shapes the text (positioning combining marks at the
same x coordinate as their base character), the force-width loop blindly
increments the cell position counter for **every** glyph — including
zero-advance combining marks. This displaces them into the next cell,
breaking the visual output.

For example, Thai `สวัสดี` renders as `ส ว ั ส ด ี` with vowels and tone
marks scattered across separate cells.

zed-industries#39526 previously fixed Latin combining characters (like NFD-normalized
é) by handling them at the terminal cell level via
`append_zero_width_chars`. That fix works when alacritty's cell model
already identifies the characters as zero-width. However, many complex
script combining marks still get mispositioned at the shaping layer —
which is what this PR addresses.

## Solution

Extracted `apply_force_width_to_layout()` in `line_layout.rs` that
detects combining marks during the force-width pass by checking whether
a glyph's shaped x position has advanced by at least half a cell width
beyond the previous base character. Combining marks (which HarfBuzz
places at the same x as their base) are kept at their shaped position
relative to the forced base, rather than being pushed to the next cell.

This approach is **script-agnostic** — it works for all complex scripts
(Thai, Arabic, Devanagari, Myanmar, Khmer, etc.) without hardcoding
Unicode ranges or disabling the monospace grid. The fix lives at the
text shaping layer in GPUI, so it benefits any consumer of
`LineLayoutCache` that uses `force_width`.

The two duplicate force-width loops (in `layout_line` and
`layout_line_by_hash`) are consolidated into the single shared helper.

## Testing

**Unit tests** cover the following scenarios:
- Latin text positions remain unchanged
- A single combining mark stays at its base character's cell
- A base character following combining marks gets the correct next cell
- Multiple stacked combining marks all stay anchored to cell 0
- Drifted base positions respect the existing 1px tolerance

**Manual testing** with the following scripts in the integrated
terminal:

| Script | Test text | What to look for |
|--------|-----------|-----------------|
| Thai | `สวัสดีครับ ภาษาไทย ก้อนหิน น้ำตก` | Vowel signs (ั ี) and tone
marks (้) render above/below consonants, not in separate cells |
| Lao | `ສະບາຍດີ ພາສາລາວ` | Vowel signs stay attached to consonants |
| Arabic | `بِسْمِ اللَّهِ الرَّحْمَٰنِ` | Diacritics (harakat) render
on their base letters |
| Hebrew | `בְּרֵאשִׁית בָּרָא אֱלֹהִים` | Vowel points (nikud) stay
under/over their letters |
| Hindi | `नमस्ते हिन्दी भाषा` | Vowel matras render correctly on
consonants |
| Bengali | `বাংলা ভাষা নমস্কার` | Combining marks stay attached |
| Tamil | `தமிழ் மொழி வணக்கம்` | Vowel signs render correctly |
| Myanmar | `မြန်မာဘာသာ မင်္ဂလာပါ` | Combining marks and medial
consonants stay in place |
| Khmer | `ភាសាខ្មែរ សួស្តី` | Subscript consonants and vowel signs
render correctly |

<details>
<summary>Script to reproduce</summary>

```sh
echo "=== Thai ==="
echo "สวัสดีครับ ภาษาไทย ก้อนหิน น้ำตก"
echo ""
echo "=== Lao ==="
echo "ສະບາຍດີ ພາສາລາວ"
echo ""
echo "=== Arabic ==="
echo "بِسْمِ اللَّهِ الرَّحْمَٰنِ"
echo ""
echo "=== Hebrew ==="
echo "בְּרֵאשִׁית בָּרָא אֱלֹהִים"
echo ""
echo "=== Hindi (Devanagari) ==="
echo "नमस्ते हिन्दी भाषा"
echo ""
echo "=== Bengali ==="
echo "বাংলা ভাষা নমস্কার"
echo ""
echo "=== Tamil ==="
echo "தமிழ் மொழி வணக்கம்"
echo ""
echo "=== Myanmar ==="
echo "မြန်မာဘာသာ မင်္ဂလာပါ"
echo ""
echo "=== Khmer ==="
echo "ភាសាខ្មែរ សួស្តី"
```

</details>

### Before

<img width="1144" height="740" alt="Screenshot 2569-04-05 at 10 43 00"
src="https://github.com/user-attachments/assets/bce2075a-9e19-40dd-81ea-0e29a451b781"
/>

### After

<img width="1144" height="747" alt="Screenshot 2569-04-05 at 11 22 55"
src="https://github.com/user-attachments/assets/e0613bf6-8452-4c65-b893-f9931a471b2e"
/>

### Tests passing

<img width="1144" height="740" alt="Screenshot 2569-04-05 at 11 20 22"
src="https://github.com/user-attachments/assets/9100bd77-4ea6-4f77-ae31-5129484552fd"
/>

Release Notes:

- Fixed terminal rendering of combining marks in complex scripts (Thai,
Arabic, Hebrew, Devanagari, Myanmar, Khmer, and others) where vowel
signs and tone marks were incorrectly displaced to adjacent cells
instead of rendering on their base characters.

---------

Co-authored-by: Conrad Irwin <conrad.irwin@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla-signed The user has signed the Contributor License Agreement first contribution the author's first pull request to Zed. NOTE: the label application is automated via github actions

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Terminal: Incorrect Thai character rendering (improper spacing and text shaping) on Windows

8 participants