Skip to content

fix: Installation tests by pinning lm-eval extras.#585

Open
ParagEkbote wants to merge 1 commit intoPrunaAI:mainfrom
ParagEkbote:fix-installation-tests
Open

fix: Installation tests by pinning lm-eval extras.#585
ParagEkbote wants to merge 1 commit intoPrunaAI:mainfrom
ParagEkbote:fix-installation-tests

Conversation

@ParagEkbote
Copy link
Copy Markdown
Contributor

@ParagEkbote ParagEkbote commented Mar 22, 2026

Description

Installation tests have been failing on main due to missing wheels for nvidia-cutlass-dsl-libs-base and nvidia-cudnn-frontend since both of these deps publish wheels only for linux, not for MacOS and Windows. Could you please review?

cc: @johannaSommer

ref:
https://github.com/PrunaAI/pruna/actions/runs/23354770495/job/67942686570
https://github.com/PrunaAI/pruna/actions/runs/23354770495/job/67942686541

Related Issue

Fixes #(issue number)

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

How Has This Been Tested?

Checklist

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

Additional Notes

@ParagEkbote ParagEkbote changed the title fix: Installation tests by pinning vLLM extras. fix: Installation tests by pinning lm-harness extras. Mar 22, 2026
@ParagEkbote ParagEkbote changed the title fix: Installation tests by pinning lm-harness extras. fix: Installation tests by pinning lm-eval extras. Mar 22, 2026
@ParagEkbote
Copy link
Copy Markdown
Contributor Author

I'm a bit perplexed about the deps either being extras of vllm or lm-eval. Could you please let me know?

cc: @johannaSommer

@sdiazlor
Copy link
Copy Markdown
Collaborator

Hi @ParagEkbote! Thanks for the quick review. We'll check asap! It should solve the failings.

@davidberenstein1957
Copy link
Copy Markdown
Member

Closed and re-opened to re-trigger the tests :)

@github-actions
Copy link
Copy Markdown

This PR has been inactive for 10 days and is now marked as stale.

@github-actions github-actions bot added the stale label Apr 12, 2026
vllm = [
"vllm>=0.16.0",
"ray",
"nvidia-cudnn-frontend; sys_platform == 'linux'",
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am a bit confused, why did we change the vllm extras, for fixing the lm-eval problem?

Copy link
Copy Markdown
Contributor Author

@ParagEkbote ParagEkbote Apr 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, as you can view in the failing jobs, there are a couple of missing deps which are not installed during the installation test, which may cause the test to fail. Attaching some images for more context:

Image 1:
image

Image 2:
image

So, I am a bit uncertain if the deps should be a part of vllm or lm-eval dependency grouping since these are cuda driver packages which are generally pulled for vllm. I'd be happy to iterate to change them for lm-eval dependency grouping.

@github-actions github-actions bot removed the stale label Apr 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants