Skip to content

Use PurePosixPath for cross-platform path handling - #15238

Merged
blisc merged 4 commits into
NVIDIA-NeMo:mainfrom
yurekami:fix/cross-platform-path
Jan 14, 2026
Merged

Use PurePosixPath for cross-platform path handling#15238
blisc merged 4 commits into
NVIDIA-NeMo:mainfrom
yurekami:fix/cross-platform-path

Conversation

@yurekami

@yurekami yurekami commented Dec 29, 2025

Copy link
Copy Markdown
Contributor

Summary

Fixes #15113

Changed path parsing to use PurePosixPath instead of str.split("/") for extracting filenames from:

  • Cloud storage URLs (s3://, gs://, etc.)
  • Hugging Face repository names

This ensures consistent behavior across platforms and properly handles edge cases.

Files modified:

  • nemo/core/connectors/save_restore_connector.py
  • nemo/core/classes/common.py

Test plan

  • Verified syntax check passes
  • Path extraction works correctly for cloud URLs and HF repo names

@github-actions github-actions Bot added the core Changes to NeMo Core label Dec 29, 2025
@yurekami
yurekami force-pushed the fix/cross-platform-path branch from e453be5 to 0bee50a Compare December 29, 2025 14:29
Replace string split("/") with PurePosixPath().name for extracting
filenames from cloud storage paths, URLs, and HuggingFace repo names.

While these paths use forward slashes by convention (not affected by
the host OS), using PurePosixPath is more explicit, self-documenting,
and follows Python best practices for path manipulation.

Files modified:
- nemo/core/connectors/save_restore_connector.py (cloud storage paths)
- nemo/core/classes/common.py (cloud URLs and HuggingFace repo names)

Note: tarutils.py and nemo_file.py were NOT modified as they handle
tar archive internal paths (which mandate POSIX format) and dictionary
keys (not filesystem paths) respectively.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: yurekami <yurekami@users.noreply.github.com>
@yurekami
yurekami force-pushed the fix/cross-platform-path branch from 0bee50a to 3c2782d Compare December 29, 2025 14:32
Signed-off-by: Jason <jasoli@nvidia.com>
blisc
blisc previously approved these changes Jan 12, 2026
Signed-off-by: Jason <jasoli@nvidia.com>
Signed-off-by: blisc <blisc@users.noreply.github.com>
@blisc
blisc enabled auto-merge (squash) January 13, 2026 16:56
@blisc blisc added the Run CICD label Jan 13, 2026
@blisc
blisc merged commit 798f0c6 into NVIDIA-NeMo:main Jan 14, 2026
253 checks passed
AkCodes23 pushed a commit to AkCodes23/NeMo that referenced this pull request Jan 28, 2026
* Use PurePosixPath for cross-platform path handling (NVIDIA-NeMo#15113)

Replace string split("/") with PurePosixPath().name for extracting
filenames from cloud storage paths, URLs, and HuggingFace repo names.

While these paths use forward slashes by convention (not affected by
the host OS), using PurePosixPath is more explicit, self-documenting,
and follows Python best practices for path manipulation.

Files modified:
- nemo/core/connectors/save_restore_connector.py (cloud storage paths)
- nemo/core/classes/common.py (cloud URLs and HuggingFace repo names)

Note: tarutils.py and nemo_file.py were NOT modified as they handle
tar archive internal paths (which mandate POSIX format) and dictionary
keys (not filesystem paths) respectively.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: yurekami <yurekami@users.noreply.github.com>

* Update common.py

Signed-off-by: Jason <jasoli@nvidia.com>

* Apply isort and black reformatting

Signed-off-by: blisc <blisc@users.noreply.github.com>

---------

Signed-off-by: yurekami <yurekami@users.noreply.github.com>
Signed-off-by: Jason <jasoli@nvidia.com>
Signed-off-by: blisc <blisc@users.noreply.github.com>
Co-authored-by: yurekami <yurekami@users.noreply.github.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Jason <jasoli@nvidia.com>
Co-authored-by: blisc <blisc@users.noreply.github.com>
Signed-off-by: Akhil Varanasi <akhilvaranasi23@gmail.com>
nune-tadevosyan pushed a commit to nune-tadevosyan/NeMo that referenced this pull request Mar 13, 2026
* Use PurePosixPath for cross-platform path handling (NVIDIA-NeMo#15113)

Replace string split("/") with PurePosixPath().name for extracting
filenames from cloud storage paths, URLs, and HuggingFace repo names.

While these paths use forward slashes by convention (not affected by
the host OS), using PurePosixPath is more explicit, self-documenting,
and follows Python best practices for path manipulation.

Files modified:
- nemo/core/connectors/save_restore_connector.py (cloud storage paths)
- nemo/core/classes/common.py (cloud URLs and HuggingFace repo names)

Note: tarutils.py and nemo_file.py were NOT modified as they handle
tar archive internal paths (which mandate POSIX format) and dictionary
keys (not filesystem paths) respectively.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: yurekami <yurekami@users.noreply.github.com>

* Update common.py

Signed-off-by: Jason <jasoli@nvidia.com>

* Apply isort and black reformatting

Signed-off-by: blisc <blisc@users.noreply.github.com>

---------

Signed-off-by: yurekami <yurekami@users.noreply.github.com>
Signed-off-by: Jason <jasoli@nvidia.com>
Signed-off-by: blisc <blisc@users.noreply.github.com>
Co-authored-by: yurekami <yurekami@users.noreply.github.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Jason <jasoli@nvidia.com>
Co-authored-by: blisc <blisc@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core Changes to NeMo Core

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Fix cross-platform path handling using string splitting instead of pathlib

2 participants