Skip to content

[algo] fix: vf_loss factor #2016

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
tongyx361 merged 3 commits into from
Jun 14, 2025
Merged

[algo] fix: vf_loss factor #2016

tongyx361 merged 3 commits into from
Jun 14, 2025

Conversation

@tongyx361
Copy link
Collaborator

@tongyx361 tongyx361 commented Jun 14, 2025 •
edited
Loading

Checklist Before Starting

  • Searched for similar PR(s).
  • Checked PR Title format
    • In format of: [modules] type: Title
    • modules are in fsdp, megatron, sglang, vllm, rollout, trainer, ci, training_utils, recipe, hardware, deployment, ray, worker, single_controller, misc, perf, model, algo, env, tool, ckpt, doc
    • type is in feat, fix, refactor, chore
    • can involve multiple modules, seperated by , or space, like [megatron, fsdp, doc] feat: xxx

What does this PR do?

- __all__ = ["register", "get_adv_estimator_fn", "AdvantageEstimator"]
+ __all__ = ["register_adv_est", "get_adv_estimator_fn", "AdvantageEstimator"]

Checklist Before Submitting

  • Read the Contribute Guide.
  • Apply pre-commit checks.
  • Add [BREAKING] to the PR title description if it breaks any API.
  • Update the documentation about your changes in the docs.
  • New CI unit test(s) are added to cover the code path.
  • Rely on existing unit tests on CI that covers the code path.

vermouth1992
vermouth1992 previously approved these changes Jun 14, 2025
View reviewed changes
eric-haibin-lin
eric-haibin-lin reviewed Jun 14, 2025
View reviewed changes
@tongyx361 tongyx361 enabled auto-merge (squash) June 14, 2025 03:39
@tongyx361 tongyx361 disabled auto-merge June 14, 2025 03:39
@tongyx361 tongyx361 enabled auto-merge (squash) June 14, 2025 03:39
@tongyx361 tongyx361 merged commit 6e15bbe into volcengine:main Jun 14, 2025
35 of 37 checks passed
yellowbee686 pushed a commit to yellowbee686/verl that referenced this pull request Jun 18, 2025
@tongyx361
[algo] fix: vf_loss factor (volcengine#2016)
09c07b3 
### Checklist Before Starting

- [x] Searched for similar PR(s).
- [x] Checked PR Title format
  - In format of: [modules] type: Title
- modules are in `fsdp, megatron, sglang, vllm, rollout, trainer, ci,
training_utils, recipe, hardware, deployment, ray, worker,
single_controller, misc, perf, model, algo, env, tool, ckpt, doc`
  - type is in `feat, fix, refactor, chore`
- can involve multiple modules, seperated by `,` or space, like
`[megatron, fsdp, doc] feat: xxx`

### What does this PR do?

- Fix `vf_loss` factor:
volcengine@ae528e0#diff-af3da2c60785abde478f7bb68c303cd20e044e8af1b1ae93a2698f5b8fd5ed63R646-R647
- Fix `core_algos.__all__`:

```diff
- __all__ = ["register", "get_adv_estimator_fn", "AdvantageEstimator"]
+ __all__ = ["register_adv_est", "get_adv_estimator_fn", "AdvantageEstimator"]
```

### Checklist Before Submitting

- [x] Read the [Contribute
Guide](https://github.com/volcengine/verl?tab=readme-ov-file#contribution-guide).
- [x] Apply [pre-commit
checks](https://github.com/volcengine/verl?tab=readme-ov-file#code-linting-and-formatting).
- [x] Add `[BREAKING]` to the PR title `description` if it breaks any
API.
- [x] Update the documentation about your changes in the
[docs](https://github.com/volcengine/verl/tree/main/docs).
- [x] New CI unit test(s) are added to cover the code path.
- [x] Rely on existing unit tests on CI that covers the code path.
Tyizhanshen pushed a commit to HyperdriveHustle/verl that referenced this pull request Jul 1, 2025
@tongyx361
[algo] fix: vf_loss factor (volcengine#2016)
7c8c817 
### Checklist Before Starting

- [x] Searched for similar PR(s).
- [x] Checked PR Title format
  - In format of: [modules] type: Title
- modules are in `fsdp, megatron, sglang, vllm, rollout, trainer, ci,
training_utils, recipe, hardware, deployment, ray, worker,
single_controller, misc, perf, model, algo, env, tool, ckpt, doc`
  - type is in `feat, fix, refactor, chore`
- can involve multiple modules, seperated by `,` or space, like
`[megatron, fsdp, doc] feat: xxx`

### What does this PR do?

- Fix `vf_loss` factor:
volcengine@ae528e0#diff-af3da2c60785abde478f7bb68c303cd20e044e8af1b1ae93a2698f5b8fd5ed63R646-R647
- Fix `core_algos.__all__`:

```diff
- __all__ = ["register", "get_adv_estimator_fn", "AdvantageEstimator"]
+ __all__ = ["register_adv_est", "get_adv_estimator_fn", "AdvantageEstimator"]
```

### Checklist Before Submitting

- [x] Read the [Contribute
Guide](https://github.com/volcengine/verl?tab=readme-ov-file#contribution-guide).
- [x] Apply [pre-commit
checks](https://github.com/volcengine/verl?tab=readme-ov-file#code-linting-and-formatting).
- [x] Add `[BREAKING]` to the PR title `description` if it breaks any
API.
- [x] Update the documentation about your changes in the
[docs](https://github.com/volcengine/verl/tree/main/docs).
- [x] New CI unit test(s) are added to cover the code path.
- [x] Rely on existing unit tests on CI that covers the code path.
whatadayG pushed a commit to whatadayG/verl that referenced this pull request Sep 5, 2025
@tongyx361
[algo] fix: vf_loss factor (volcengine#2016)
f67d030 
### Checklist Before Starting

- [x] Searched for similar PR(s).
- [x] Checked PR Title format
  - In format of: [modules] type: Title
- modules are in `fsdp, megatron, sglang, vllm, rollout, trainer, ci,
training_utils, recipe, hardware, deployment, ray, worker,
single_controller, misc, perf, model, algo, env, tool, ckpt, doc`
  - type is in `feat, fix, refactor, chore`
- can involve multiple modules, seperated by `,` or space, like
`[megatron, fsdp, doc] feat: xxx`

### What does this PR do?

- Fix `vf_loss` factor:
volcengine@ae528e0#diff-af3da2c60785abde478f7bb68c303cd20e044e8af1b1ae93a2698f5b8fd5ed63R646-R647
- Fix `core_algos.__all__`:

```diff
- __all__ = ["register", "get_adv_estimator_fn", "AdvantageEstimator"]
+ __all__ = ["register_adv_est", "get_adv_estimator_fn", "AdvantageEstimator"]
```

### Checklist Before Submitting

- [x] Read the [Contribute
Guide](https://github.com/volcengine/verl?tab=readme-ov-file#contribution-guide).
- [x] Apply [pre-commit
checks](https://github.com/volcengine/verl?tab=readme-ov-file#code-linting-and-formatting).
- [x] Add `[BREAKING]` to the PR title `description` if it breaks any
API.
- [x] Update the documentation about your changes in the
[docs](https://github.com/volcengine/verl/tree/main/docs).
- [x] New CI unit test(s) are added to cover the code path.
- [x] Rely on existing unit tests on CI that covers the code path.
chenjiaoAngel added a commit to chenjiaoAngel/verl that referenced this pull request Nov 14, 2025
@chenjiaoAngel
[algo] fix: vf_loss factor (volcengine#2016)
37d3544 
### Checklist Before Starting

- [x] Searched for similar PR(s).
- [x] Checked PR Title format
  - In format of: [modules] type: Title
- modules are in `fsdp, megatron, sglang, vllm, rollout, trainer, ci,
training_utils, recipe, hardware, deployment, ray, worker,
single_controller, misc, perf, model, algo, env, tool, ckpt, doc`
  - type is in `feat, fix, refactor, chore`
- can involve multiple modules, seperated by `,` or space, like
`[megatron, fsdp, doc] feat: xxx`

### What does this PR do?

- Fix `vf_loss` factor:
volcengine@ae528e0#diff-af3da2c60785abde478f7bb68c303cd20e044e8af1b1ae93a2698f5b8fd5ed63R646-R647
- Fix `core_algos.__all__`:

```diff
- __all__ = ["register", "get_adv_estimator_fn", "AdvantageEstimator"]
+ __all__ = ["register_adv_est", "get_adv_estimator_fn", "AdvantageEstimator"]
```

### Checklist Before Submitting

- [x] Read the [Contribute
Guide](https://github.com/volcengine/verl?tab=readme-ov-file#contribution-guide).
- [x] Apply [pre-commit
checks](https://github.com/volcengine/verl?tab=readme-ov-file#code-linting-and-formatting).
- [x] Add `[BREAKING]` to the PR title `description` if it breaks any
API.
- [x] Update the documentation about your changes in the
[docs](https://github.com/volcengine/verl/tree/main/docs).
- [x] New CI unit test(s) are added to cover the code path.
- [x] Rely on existing unit tests on CI that covers the code path.
TimurTaepov pushed a commit to giorgossideris/verl that referenced this pull request Dec 20, 2025
@tongyx361
[algo] fix: vf_loss factor (volcengine#2016)
c0684b2 
### Checklist Before Starting

- [x] Searched for similar PR(s).
- [x] Checked PR Title format
  - In format of: [modules] type: Title
- modules are in `fsdp, megatron, sglang, vllm, rollout, trainer, ci,
training_utils, recipe, hardware, deployment, ray, worker,
single_controller, misc, perf, model, algo, env, tool, ckpt, doc`
  - type is in `feat, fix, refactor, chore`
- can involve multiple modules, seperated by `,` or space, like
`[megatron, fsdp, doc] feat: xxx`

### What does this PR do?

- Fix `vf_loss` factor:
volcengine@ae528e0#diff-af3da2c60785abde478f7bb68c303cd20e044e8af1b1ae93a2698f5b8fd5ed63R646-R647
- Fix `core_algos.__all__`:

```diff
- __all__ = ["register", "get_adv_estimator_fn", "AdvantageEstimator"]
+ __all__ = ["register_adv_est", "get_adv_estimator_fn", "AdvantageEstimator"]
```

### Checklist Before Submitting

- [x] Read the [Contribute
Guide](https://github.com/volcengine/verl?tab=readme-ov-file#contribution-guide).
- [x] Apply [pre-commit
checks](https://github.com/volcengine/verl?tab=readme-ov-file#code-linting-and-formatting).
- [x] Add `[BREAKING]` to the PR title `description` if it breaks any
API.
- [x] Update the documentation about your changes in the
[docs](https://github.com/volcengine/verl/tree/main/docs).
- [x] New CI unit test(s) are added to cover the code path.
- [x] Rely on existing unit tests on CI that covers the code path.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@eric-haibin-lin eric-haibin-lin eric-haibin-lin left review comments

@vermouth1992 vermouth1992 vermouth1992 left review comments

@hiyouga hiyouga hiyouga approved these changes

@PeterSH6 PeterSH6 Awaiting requested review from PeterSH6 PeterSH6 is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@tongyx361 @eric-haibin-lin @vermouth1992 @hiyouga