Correctly pass gen_kwarg to eval during model runs #5451
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
Author
|
Hi @hiyouga mind reviewing this PR? |
hiyouga
force-pushed
the
fix-gen-kwarg
branch
from
d1b0f6c to
6eb7ece
Compare
hiyouga
approved these changes
Feb 12, 2025
hiyouga
added
solved
This was referenced Feb 13, 2025
1587causalai
pushed a commit
to 1587causalai/llama_factory
that referenced
this pull request
Feb 18, 2025
* Correctly pass gen_kwarg to eval during model runs * fix * fix --------- Co-authored-by: hiyouga <[email protected]>
This was referenced Feb 19, 2025
This was referenced Feb 27, 2025
This was referenced Mar 6, 2025
stephen-nju
pushed a commit
to stephen-nju/Llmtrain
that referenced
this pull request
Mar 24, 2025
* Correctly pass gen_kwarg to eval during model runs * fix * fix --------- Co-authored-by: hiyouga <[email protected]> Former-commit-id: 11eac71
yoonseok312
pushed a commit
to pensieve-ai/LLaMA-Factory-vlm
that referenced
this pull request
Apr 29, 2025
* Correctly pass gen_kwarg to eval during model runs * fix * fix --------- Co-authored-by: hiyouga <[email protected]> Former-commit-id: 11eac71
liu-qingyuan
pushed a commit
to liu-qingyuan/LLaMA-Factory-Megafake
that referenced
this pull request
Jun 6, 2025
* Correctly pass gen_kwarg to eval during model runs * fix * fix --------- Co-authored-by: hiyouga <[email protected]> Former-commit-id: 11eac71
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Fixes #5444
The current impl disregards whatever user might have set in the yaml config during evaluations that happen during training (as specified by user e.g. every 100 step eval once)
Instead, it is only applied correctly at the final eval after training is done.
This should not be the case as that would make any intermediate eval results inaccurate.
Before submitting