Fix terminal combining marks - #53176
Merged
ConradIrwin merged 5 commits intoApr 23, 2026
Merged
Conversation
The force_width loop in LineLayoutCache incorrectly incremented the cell position counter for every glyph, including combining marks (zero-advance glyphs like Thai vowel signs and tone marks). This caused combining marks to be displaced to the next cell instead of rendering on top of their base character. Detect combining marks by checking whether the shaped x position has advanced by at least half a cell beyond the previous base character. Unlike script-specific fixes, this handles all complex scripts (Thai, Arabic, Devanagari, etc.) at the text shaping layer. Closes zed-industries#46942
Tests verify that: - Latin text positions are unchanged - Combining marks stay at their base character's position - Base characters after combining marks get the correct cell - Multiple stacked combining marks all stay at cell 0 - Drifted base positions are corrected to cell boundaries
- Fix combining mark misalignment when base glyph is within 1px tolerance: track actual glyph position instead of grid slot - Extract glyph_x_positions test helper to eliminate repetition - Use f32::from() instead of direct .0 field access - Remove redundant comments that restate assertions
zed-codeowner-coordinator
Bot
requested review from
a team,
as-cii and
dinocosta
and removed request for
a team
April 5, 2026 04:29
Contributor
Author
|
Hi @as-cii @dinocosta 🙏 I've submitted this PR to fix the terminal combining marks issue (especially affecting Thai, Lao, Arabic, and other complex scripts where diacritics/vowels are rendered in wrong positions). This is important for me ; w ; please consider Thank you very much! |
Member
|
Interesting. I had expected we'd want to fix this using something like the unicode segmentation crate to do this, but this seems to work effectively and avoids drift between the text renderer and our hypotheticals. Willing to give it a go. Thanks for the examples. |
ConradIrwin
approved these changes
Apr 23, 2026
ConradIrwin
enabled auto-merge (squash)
April 23, 2026 02:39
AzureZee
added a commit
to AzureZee/gpuix
that referenced
this pull request
Apr 23, 2026
AzureZee
added a commit
to AzureZee/gpuix
that referenced
this pull request
Apr 23, 2026
|
tysm 😭 |
|
ขอบคุณครับ เฝ้ารอ release นี้อย่างตั้งใจ |
Member
|
|
kathbigra
pushed a commit
to kathbigra/zed
that referenced
this pull request
May 10, 2026
Self-Review Checklist: - [x] I've reviewed my own diff for quality, security, and reliability - [x] Unsafe blocks (if any) have justifying comments - [x] The content is consistent with the [UI/UX checklist](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist) - [x] Tests cover the new/changed behavior - [x] Performance impact has been considered and is acceptable Closes zed-industries#46942 ## Problem The terminal's monospace grid forcing logic in `LineLayoutCache` breaks rendering of combining marks in complex scripts. Thai, Lao, Arabic, Hebrew, Myanmar, Khmer, and other scripts use combining characters (vowels, tone marks, diacritics) that should render on top of or below their base character — not in a separate cell. The root cause is in `apply_force_width` within `line_layout.rs`. After HarfBuzz correctly shapes the text (positioning combining marks at the same x coordinate as their base character), the force-width loop blindly increments the cell position counter for **every** glyph — including zero-advance combining marks. This displaces them into the next cell, breaking the visual output. For example, Thai `สวัสดี` renders as `ส ว ั ส ด ี` with vowels and tone marks scattered across separate cells. zed-industries#39526 previously fixed Latin combining characters (like NFD-normalized é) by handling them at the terminal cell level via `append_zero_width_chars`. That fix works when alacritty's cell model already identifies the characters as zero-width. However, many complex script combining marks still get mispositioned at the shaping layer — which is what this PR addresses. ## Solution Extracted `apply_force_width_to_layout()` in `line_layout.rs` that detects combining marks during the force-width pass by checking whether a glyph's shaped x position has advanced by at least half a cell width beyond the previous base character. Combining marks (which HarfBuzz places at the same x as their base) are kept at their shaped position relative to the forced base, rather than being pushed to the next cell. This approach is **script-agnostic** — it works for all complex scripts (Thai, Arabic, Devanagari, Myanmar, Khmer, etc.) without hardcoding Unicode ranges or disabling the monospace grid. The fix lives at the text shaping layer in GPUI, so it benefits any consumer of `LineLayoutCache` that uses `force_width`. The two duplicate force-width loops (in `layout_line` and `layout_line_by_hash`) are consolidated into the single shared helper. ## Testing **Unit tests** cover the following scenarios: - Latin text positions remain unchanged - A single combining mark stays at its base character's cell - A base character following combining marks gets the correct next cell - Multiple stacked combining marks all stay anchored to cell 0 - Drifted base positions respect the existing 1px tolerance **Manual testing** with the following scripts in the integrated terminal: | Script | Test text | What to look for | |--------|-----------|-----------------| | Thai | `สวัสดีครับ ภาษาไทย ก้อนหิน น้ำตก` | Vowel signs (ั ี) and tone marks (้) render above/below consonants, not in separate cells | | Lao | `ສະບາຍດີ ພາສາລາວ` | Vowel signs stay attached to consonants | | Arabic | `بِسْمِ اللَّهِ الرَّحْمَٰنِ` | Diacritics (harakat) render on their base letters | | Hebrew | `בְּרֵאשִׁית בָּרָא אֱלֹהִים` | Vowel points (nikud) stay under/over their letters | | Hindi | `नमस्ते हिन्दी भाषा` | Vowel matras render correctly on consonants | | Bengali | `বাংলা ভাষা নমস্কার` | Combining marks stay attached | | Tamil | `தமிழ் மொழி வணக்கம்` | Vowel signs render correctly | | Myanmar | `မြန်မာဘာသာ မင်္ဂလာပါ` | Combining marks and medial consonants stay in place | | Khmer | `ភាសាខ្មែរ សួស្តី` | Subscript consonants and vowel signs render correctly | <details> <summary>Script to reproduce</summary> ```sh echo "=== Thai ===" echo "สวัสดีครับ ภาษาไทย ก้อนหิน น้ำตก" echo "" echo "=== Lao ===" echo "ສະບາຍດີ ພາສາລາວ" echo "" echo "=== Arabic ===" echo "بِسْمِ اللَّهِ الرَّحْمَٰنِ" echo "" echo "=== Hebrew ===" echo "בְּרֵאשִׁית בָּרָא אֱלֹהִים" echo "" echo "=== Hindi (Devanagari) ===" echo "नमस्ते हिन्दी भाषा" echo "" echo "=== Bengali ===" echo "বাংলা ভাষা নমস্কার" echo "" echo "=== Tamil ===" echo "தமிழ் மொழி வணக்கம்" echo "" echo "=== Myanmar ===" echo "မြန်မာဘာသာ မင်္ဂလာပါ" echo "" echo "=== Khmer ===" echo "ភាសាខ្មែរ សួស្តី" ``` </details> ### Before <img width="1144" height="740" alt="Screenshot 2569-04-05 at 10 43 00" src="https://github.com/user-attachments/assets/bce2075a-9e19-40dd-81ea-0e29a451b781" /> ### After <img width="1144" height="747" alt="Screenshot 2569-04-05 at 11 22 55" src="https://github.com/user-attachments/assets/e0613bf6-8452-4c65-b893-f9931a471b2e" /> ### Tests passing <img width="1144" height="740" alt="Screenshot 2569-04-05 at 11 20 22" src="https://github.com/user-attachments/assets/9100bd77-4ea6-4f77-ae31-5129484552fd" /> Release Notes: - Fixed terminal rendering of combining marks in complex scripts (Thai, Arabic, Hebrew, Devanagari, Myanmar, Khmer, and others) where vowel signs and tone marks were incorrectly displaced to adjacent cells instead of rendering on their base characters. --------- Co-authored-by: Conrad Irwin <conrad.irwin@gmail.com>
jonx
pushed a commit
to jonx/zed-aros
that referenced
this pull request
Jul 17, 2026
Self-Review Checklist: - [x] I've reviewed my own diff for quality, security, and reliability - [x] Unsafe blocks (if any) have justifying comments - [x] The content is consistent with the [UI/UX checklist](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist) - [x] Tests cover the new/changed behavior - [x] Performance impact has been considered and is acceptable Closes zed-industries#46942 ## Problem The terminal's monospace grid forcing logic in `LineLayoutCache` breaks rendering of combining marks in complex scripts. Thai, Lao, Arabic, Hebrew, Myanmar, Khmer, and other scripts use combining characters (vowels, tone marks, diacritics) that should render on top of or below their base character — not in a separate cell. The root cause is in `apply_force_width` within `line_layout.rs`. After HarfBuzz correctly shapes the text (positioning combining marks at the same x coordinate as their base character), the force-width loop blindly increments the cell position counter for **every** glyph — including zero-advance combining marks. This displaces them into the next cell, breaking the visual output. For example, Thai `สวัสดี` renders as `ส ว ั ส ด ี` with vowels and tone marks scattered across separate cells. zed-industries#39526 previously fixed Latin combining characters (like NFD-normalized é) by handling them at the terminal cell level via `append_zero_width_chars`. That fix works when alacritty's cell model already identifies the characters as zero-width. However, many complex script combining marks still get mispositioned at the shaping layer — which is what this PR addresses. ## Solution Extracted `apply_force_width_to_layout()` in `line_layout.rs` that detects combining marks during the force-width pass by checking whether a glyph's shaped x position has advanced by at least half a cell width beyond the previous base character. Combining marks (which HarfBuzz places at the same x as their base) are kept at their shaped position relative to the forced base, rather than being pushed to the next cell. This approach is **script-agnostic** — it works for all complex scripts (Thai, Arabic, Devanagari, Myanmar, Khmer, etc.) without hardcoding Unicode ranges or disabling the monospace grid. The fix lives at the text shaping layer in GPUI, so it benefits any consumer of `LineLayoutCache` that uses `force_width`. The two duplicate force-width loops (in `layout_line` and `layout_line_by_hash`) are consolidated into the single shared helper. ## Testing **Unit tests** cover the following scenarios: - Latin text positions remain unchanged - A single combining mark stays at its base character's cell - A base character following combining marks gets the correct next cell - Multiple stacked combining marks all stay anchored to cell 0 - Drifted base positions respect the existing 1px tolerance **Manual testing** with the following scripts in the integrated terminal: | Script | Test text | What to look for | |--------|-----------|-----------------| | Thai | `สวัสดีครับ ภาษาไทย ก้อนหิน น้ำตก` | Vowel signs (ั ี) and tone marks (้) render above/below consonants, not in separate cells | | Lao | `ສະບາຍດີ ພາສາລາວ` | Vowel signs stay attached to consonants | | Arabic | `بِسْمِ اللَّهِ الرَّحْمَٰنِ` | Diacritics (harakat) render on their base letters | | Hebrew | `בְּרֵאשִׁית בָּרָא אֱלֹהִים` | Vowel points (nikud) stay under/over their letters | | Hindi | `नमस्ते हिन्दी भाषा` | Vowel matras render correctly on consonants | | Bengali | `বাংলা ভাষা নমস্কার` | Combining marks stay attached | | Tamil | `தமிழ் மொழி வணக்கம்` | Vowel signs render correctly | | Myanmar | `မြန်မာဘာသာ မင်္ဂလာပါ` | Combining marks and medial consonants stay in place | | Khmer | `ភាសាខ្មែរ សួស្តី` | Subscript consonants and vowel signs render correctly | <details> <summary>Script to reproduce</summary> ```sh echo "=== Thai ===" echo "สวัสดีครับ ภาษาไทย ก้อนหิน น้ำตก" echo "" echo "=== Lao ===" echo "ສະບາຍດີ ພາສາລາວ" echo "" echo "=== Arabic ===" echo "بِسْمِ اللَّهِ الرَّحْمَٰنِ" echo "" echo "=== Hebrew ===" echo "בְּרֵאשִׁית בָּרָא אֱלֹהִים" echo "" echo "=== Hindi (Devanagari) ===" echo "नमस्ते हिन्दी भाषा" echo "" echo "=== Bengali ===" echo "বাংলা ভাষা নমস্কার" echo "" echo "=== Tamil ===" echo "தமிழ் மொழி வணக்கம்" echo "" echo "=== Myanmar ===" echo "မြန်မာဘာသာ မင်္ဂလာပါ" echo "" echo "=== Khmer ===" echo "ភាសាខ្មែរ សួស្តី" ``` </details> ### Before <img width="1144" height="740" alt="Screenshot 2569-04-05 at 10 43 00" src="https://github.com/user-attachments/assets/bce2075a-9e19-40dd-81ea-0e29a451b781" /> ### After <img width="1144" height="747" alt="Screenshot 2569-04-05 at 11 22 55" src="https://github.com/user-attachments/assets/e0613bf6-8452-4c65-b893-f9931a471b2e" /> ### Tests passing <img width="1144" height="740" alt="Screenshot 2569-04-05 at 11 20 22" src="https://github.com/user-attachments/assets/9100bd77-4ea6-4f77-ae31-5129484552fd" /> Release Notes: - Fixed terminal rendering of combining marks in complex scripts (Thai, Arabic, Hebrew, Devanagari, Myanmar, Khmer, and others) where vowel signs and tone marks were incorrectly displaced to adjacent cells instead of rendering on their base characters. --------- Co-authored-by: Conrad Irwin <conrad.irwin@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Self-Review Checklist:
Closes #46942
Problem
The terminal's monospace grid forcing logic in
LineLayoutCachebreaks rendering of combining marks in complex scripts. Thai, Lao, Arabic, Hebrew, Myanmar, Khmer, and other scripts use combining characters (vowels, tone marks, diacritics) that should render on top of or below their base character — not in a separate cell.The root cause is in
apply_force_widthwithinline_layout.rs. After HarfBuzz correctly shapes the text (positioning combining marks at the same x coordinate as their base character), the force-width loop blindly increments the cell position counter for every glyph — including zero-advance combining marks. This displaces them into the next cell, breaking the visual output.For example, Thai
สวัสดีrenders asส ว ั ส ด ีwith vowels and tone marks scattered across separate cells.#39526 previously fixed Latin combining characters (like NFD-normalized é) by handling them at the terminal cell level via
append_zero_width_chars. That fix works when alacritty's cell model already identifies the characters as zero-width. However, many complex script combining marks still get mispositioned at the shaping layer — which is what this PR addresses.Solution
Extracted
apply_force_width_to_layout()inline_layout.rsthat detects combining marks during the force-width pass by checking whether a glyph's shaped x position has advanced by at least half a cell width beyond the previous base character. Combining marks (which HarfBuzz places at the same x as their base) are kept at their shaped position relative to the forced base, rather than being pushed to the next cell.This approach is script-agnostic — it works for all complex scripts (Thai, Arabic, Devanagari, Myanmar, Khmer, etc.) without hardcoding Unicode ranges or disabling the monospace grid. The fix lives at the text shaping layer in GPUI, so it benefits any consumer of
LineLayoutCachethat usesforce_width.The two duplicate force-width loops (in
layout_lineandlayout_line_by_hash) are consolidated into the single shared helper.Testing
Unit tests cover the following scenarios:
Manual testing with the following scripts in the integrated terminal:
สวัสดีครับ ภาษาไทย ก้อนหิน น้ำตกສະບາຍດີ ພາສາລາວبِسْمِ اللَّهِ الرَّحْمَٰنِבְּרֵאשִׁית בָּרָא אֱלֹהִיםनमस्ते हिन्दी भाषाবাংলা ভাষা নমস্কারதமிழ் மொழி வணக்கம்မြန်မာဘာသာ မင်္ဂလာပါភាសាខ្មែរ សួស្តីScript to reproduce
Before
After
Tests passing
Release Notes: