fix: add encoding='utf-8' to all text-mode open() calls - #8003
Open
haroldfabla2-hue wants to merge 1 commit into
Open
fix: add encoding='utf-8' to all text-mode open() calls#8003haroldfabla2-hue wants to merge 1 commit into
haroldfabla2-hue wants to merge 1 commit into
Conversation
Fixes UnicodeDecodeError on non-English systems (e.g. cp950/Big5 on Taiwan,
cp932 on Japan) where the default codec can't decode UTF-8 content in
bundled JS files, config files, session recordings, and HTML logs.
Changes across 20 files in autogen-ext, autogen-studio, agbench, and
magentic-one-cli packages:
- Add encoding='utf-8' to all text-mode ('r', 'w', 'rt', 'wt') open() calls
- Binary mode ('rb', 'wb') calls intentionally left unchanged
- Fixes microsoft#5566
Contributor
|
@haroldfabla2-hue please read the following Contributor License Agreement(CLA). If you agree with the CLA, please reply with the following information.
Contributor License AgreementContribution License AgreementThis Contribution License Agreement (“Agreement”) is agreed to by the party signing below (“You”),
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fix for #5566
Problem
On systems with non-UTF-8 default encodings (e.g.
cp950on Traditional Chinese Windows,cp932on Japanese Windows),open()calls without explicitencodinguse the locale default. This causesUnicodeDecodeErrorwhen reading UTF-8 content like bundled JS files, config files, or session recordings.Root Cause
The original bug report (#5566) identified
playwright_controller.py— that specific call was already fixed inmain. However, as the reporter noted: "there will be some similar issues in code base while using open function." This PR addresses all remaining text-modeopen()calls.Changes
Added
encoding="utf-8"to 49 text-modeopen()calls across 20 files in 4 packages:magentic-one-cli_m1.py) — the exact entry point from the original bug reportautogen-extchat_completion_client_recorder.py(session record/replay),page_logger.py(HTML logs),mcp_session_host_example.py,_docker_jupyter.pyautogen-studioschema_manager.py,auth/manager.py,gallery/builder.py,cli.py,lite/studio.pyagbenchrun_cmd.py,tabulate_cmd.py,linter/, benchmark scriptsBinary mode (
"rb","wb") calls were intentionally left unchanged since encoding does not apply.Verification
Testing
ruff checkpasses (formatting preserved)Fixes #5566