Skip to content

llama converter is self-contained now (no dependency on internal nvidia code)#552

Merged
danielkorzekwa merged 31 commits intofeature/compressfrom
dkorzekwa/llama_converter_selfcontained_2
Nov 14, 2025
Merged

llama converter is self-contained now (no dependency on internal nvidia code)#552
danielkorzekwa merged 31 commits intofeature/compressfrom
dkorzekwa/llama_converter_selfcontained_2

Conversation

@danielkorzekwa
Copy link
Copy Markdown
Contributor

@danielkorzekwa danielkorzekwa commented Nov 13, 2025

What does this PR do?

llama converter is self-contained now (no dependency on internal nvidia code)

@danielkorzekwa
Add decilm modelling code
694c317 
Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
@danielkorzekwa
Add decilm modelling code.
991659f 
Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
@danielkorzekwa
Add transformers codebase
8489cee 
Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
@danielkorzekwa
Add transformers code
f0afefe 
Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
@danielkorzekwa
Add decilm modelling code
b3ed5bc 
Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
@danielkorzekwa
Add decilm modelling code
a700da5 
Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
@danielkorzekwa
Correct licence headers
b59b679 
Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
@danielkorzekwa
Correct licence headers
1abdf3e 
Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
@danielkorzekwa
Add decilm code
66609b1 
Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
@danielkorzekwa
Add decilm code
7da0a8a 
Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
@danielkorzekwa
Add decilm code
6e09a81 
Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
@danielkorzekwa
Add decilm code
2e3f5da 
Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
@danielkorzekwa
Add decilm code
418890e 
Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
@danielkorzekwa
Make llama3 converter self-contained (no deps on internal Nvidia code)
01f4fc1 
Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
@danielkorzekwa
Add common module
c57eed4 
Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
@danielkorzekwa
module refactoring
3dc37b3 
Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
@danielkorzekwa
refactoring
10ffdfe 
Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
@danielkorzekwa
add shared_checkpointing_utils
27a4456 
Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
@danielkorzekwa
Add json tools
b0e22b7 
Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
@danielkorzekwa
add logger
52e7827 
Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
@danielkorzekwa
import refactoring
f5c1c87 
Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
@danielkorzekwa
add post_init_sparse module
0aa6320 
Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
@danielkorzekwa
Add post_init_sparse
35d0dbc 
Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
@danielkorzekwa
merginy hydra.py and hydra_utils.py
e39a1ad 
Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
@danielkorzekwa
Merge branch 'feature/compress' into dkorzekwa/llama_converter_selfco…
3f0772b 
…ntained_2

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
@danielkorzekwa danielkorzekwa requested a review from a team as a code owner November 13, 2025 17:03
@danielkorzekwa
Delete not used tokenizer
872d6c3 
Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
@codecov
Copy link
Copy Markdown

codecov bot commented Nov 13, 2025 •
edited
Loading

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 73.40%. Comparing base (b121945) to head (298050d).
⚠️ Report is 1 commits behind head on feature/compress.

Additional details and impacted files 
@@                Coverage Diff                @@
##           feature/compress     #552   +/-   ##
=================================================
  Coverage             73.40%   73.40%           
=================================================
  Files                   180      180           
  Lines                 18127    18127           
=================================================
  Hits                  13306    13306           
  Misses                 4821     4821

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

kevalmorabia97
kevalmorabia97 reviewed Nov 13, 2025
View reviewed changes
modelopt/torch/_compress/tools/sharded_checkpoint_utils.py
Copy link
Copy Markdown
Collaborator

@kevalmorabia97 kevalmorabia97 Nov 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this for pipeline parallel ckpt load and saving? Can you add a short one-line docstring to each file (after license headers) describing what it is about?

Copy link
Copy Markdown
Contributor Author

@danielkorzekwa danielkorzekwa Nov 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added

Copy link
Copy Markdown
Collaborator

@kevalmorabia97 kevalmorabia97 Nov 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So not just for PP but also for DDP?

Copy link
Copy Markdown
Contributor Author

@danielkorzekwa danielkorzekwa Nov 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not know which parallelisms are used and how precisely.

@danielkorzekwa
Refactor imports
eb60a1c 
Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
@danielkorzekwa
Improve comments + move llama convert pytest from experimental to gpu…
c1533fa 
… (it will run on CI now)

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
@danielkorzekwa
fix broken integration test
763a4d5 
Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
@danielkorzekwa
Improve pydocs
76df47c 
Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
kevalmorabia97
kevalmorabia97 reviewed Nov 14, 2025
View reviewed changes
modelopt/torch/_compress/tools/checkpoint_utils.py
Copy link
Copy Markdown
Collaborator

@kevalmorabia97 kevalmorabia97 Nov 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not for this PR, but do you think it would be better to move this in the DeciLM folder?

Copy link
Copy Markdown
Contributor Author

@danielkorzekwa danielkorzekwa Nov 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe. Actually, initially I moved it to DeciLM but then I realized in how many places it is used. Also not all logic is DeciLM specific. Given that we plan to refactor DeciLM (or even remove) it, I think it is bad timing.

kevalmorabia97
kevalmorabia97 reviewed Nov 14, 2025
View reviewed changes
@danielkorzekwa
Remove try except around import omegaconf
298050d 
Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
@AAnoosheh
Copy link
Copy Markdown
Contributor

AAnoosheh commented Nov 14, 2025

Can confirm conversion of Llama-3.2 worked

@danielkorzekwa danielkorzekwa merged commit 866e400 into feature/compress Nov 14, 2025
20 of 21 checks passed
@danielkorzekwa danielkorzekwa deleted the dkorzekwa/llama_converter_selfcontained_2 branch November 14, 2025 18:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@kevalmorabia97 kevalmorabia97 kevalmorabia97 approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@danielkorzekwa @AAnoosheh @kevalmorabia97