[Core] Make MAX_APPLICATION_ERROR_LEN configurable via env var#59543
Conversation
Currently MAX_APPLICATION_ERROR_LEN is hardcoded to 500 bytes, which truncates error messages in the dashboard. This adds RAY_MAX_APPLICATION_ERROR_LEN env var to make it configurable while keeping the 500 bytes default. Fixes ray-project#50976 Signed-off-by: Ondrej Prenek <ondra.prenek@gmail.com>
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request successfully introduces configurability for the maximum application error message length via an environment variable. This is a valuable change that addresses the issue of truncated error messages, improving debugging capabilities for users.
|
|
||
| # The max number of bytes for task execution error message. | ||
| MAX_APPLICATION_ERROR_LEN = 500 | ||
| MAX_APPLICATION_ERROR_LEN = env_integer("RAY_MAX_APPLICATION_ERROR_LEN", 500) |
There was a problem hiding this comment.
The constant MAX_APPLICATION_ERROR_LEN is explicitly described in the preceding comment (line 388) as representing "the max number of bytes". For improved clarity and consistency with other byte-related constants in this file (e.g., MACH_PAGE_SIZE_BYTES, DEFAULT_MIN_SYSTEM_RESERVED_MEMORY_BYTES), it would be more precise to rename this constant to MAX_APPLICATION_ERROR_BYTES. The corresponding environment variable RAY_MAX_APPLICATION_ERROR_LEN should also be updated to RAY_MAX_APPLICATION_ERROR_BYTES to reflect this unit.
| MAX_APPLICATION_ERROR_LEN = env_integer("RAY_MAX_APPLICATION_ERROR_LEN", 500) | |
| MAX_APPLICATION_ERROR_BYTES = env_integer("RAY_MAX_APPLICATION_ERROR_BYTES", 500) |
There was a problem hiding this comment.
not sure if we want to rename the variable, I'll leave it up to reviewer (the human one :) )
ray-gardener
Bot
added
docs
|
Let's call it |
…RROR_LENGTH to match other constants naming conventions Signed-off-by: Ondrej Prenek <ondra.prenek@gmail.com>
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
… string Signed-off-by: Ondrej Prenek <ondra.prenek@gmail.com>
|
can I ask for review, please? 🙏 |
|
@edoakes Can i ask you to release a new version so i can use it? I haven't found a way to do it myself 🙏 Thanks! |
Release should be coming in the next week or two. You can also use the nightly wheels, links here: https://docs.ray.io/en/latest/ray-overview/installation.html#daily-releases-nightlies |
…roject#59543) ## Why are these changes needed? Ray truncates error messages to 500 bytes, keeping only the last 500 characters. This often cuts off the exception type and actual error message, making debugging hard. This PR makes the limit configurable via `RAY_MAX_APPLICATION_ERROR_LEN` environment variable while keeping the 500 byte default. ## Related issue Fixes ray-project#50976 ## Changes ```python # In ray/_private/ray_constants.py MAX_APPLICATION_ERROR_LEN = env_integer("RAY_MAX_APPLICATION_ERROR_LEN", 500) ``` ## Usage ```bash # Set higher limit export RAY_MAX_APPLICATION_ERROR_LEN=5000 ray start --head ``` In Kubernetes: ```yaml env: - name: RAY_MAX_APPLICATION_ERROR_LEN value: "5000" ``` --------- Signed-off-by: Ondrej Prenek <ondra.prenek@gmail.com> Signed-off-by: jasonwrwang <jasonwrwang@tencent.com>
Hi @edoakes, just checking in — any update on the release timeline? It's been about 3 weeks since the last mention. Thanks 🙏! |
|
@preneond I apologize, our testing infra has been more unstable than usual which has delayed the release process (we run a large suite of tests prior to releasing any new versions). We are pushing hard to have a release by the end of next week at the latest. |
2 participants
Why are these changes needed?
Ray truncates error messages to 500 bytes, keeping only the last 500 characters. This often cuts off the exception type and actual error message, making debugging hard.
This PR makes the limit configurable via
RAY_MAX_APPLICATION_ERROR_LENenvironment variable while keeping the 500 byte default.Related issue
Fixes #50976
Changes
Usage
In Kubernetes: