feat: vbench datamodule

[begumcig]

Description

This PR introduces the prompt suite of VBench as a pruna datamodule.

The updates in this PR:

• Loading the VBench prompts from the JSON and turn them into datasets
• Options to load single / multi / all dimensions of the prompt suite
• Update the prompt collate to have auxiliary information that could be related to benchmarking
• Small bugfix in our utilities that changed the type of the dataset

Type of Change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

How Has This Been Tested?

Added a test in our datamodule tests

Checklist

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes

Additional Notes

You can use the new datamodule as follows:

# Load specific dimension 

dm = PrunaDataModule.from_string("VBench" , category = 'background_consistency')

# Load the entire prompt suite

dm = PrunaDataModule.from_string("VBench")

---

Note

Adds VBench prompt suite as a dataset, introduces prompt_with_auxiliaries_collate, enables category filtering via PrunaDataModule.from_string, and integrates tests.

• Datasets:
  • Add text_to_video.setup_vbench_dataset to load VBench prompts from VBench_full_info.json, rename columns to category and text, optional category filtering, and return test-focused splits.
  • Register "VBench" in base_datasets using prompt_with_auxiliaries_collate.
• Collate:
  • Introduce prompt_with_auxiliaries_collate returning List[str] prompts and auxiliary metadata dicts; expose via pruna_collate_fns.
• DataModule:
  • Extend from_string with category arg; auto-forward to setup functions that accept it.
• Tests:
  • Add VBench to test_dm_from_string matrix.

^{Written by Cursor Bugbot for commit 3d21412. This will update automatically on new commits. Configure here.}

[simlang]

Amazing! 🚀 next to typo and a docstring thing, just the collate_fn thing. completely up to you if you agree or not. easy approve

[simlang]

add to docstring

[davidberenstein1957]

Some small comments, feel free to merge after.

[davidberenstein1957]

should we mention a list of categories also somehwere in the doctstring

[begumcig]

I added a very simple definition of category here because there's a chance we will use this category attribute also for other datasets in the feature so we are not limited to VBench, but will be adding the categories of VBench in the documentation update how does that sound?

[davidberenstein1957]

you had also applied these changes in another PR. should we add some comment in either PR about whhy and what happened?

[begumcig]

Yes ofcourse! Basically, we switched to ty from mypy for static type checking and it's more restrictive than mypy. Maybe, sometime in the future we will enforce all of these rules, but for now, we are restricting them to be similar to what we had with mypy! We have this as a comment in line 22 in this file but it doesn't show since it was not added by me 🥺

[cursor]

Bug: Incorrect Iteration in List Comprehension

The list comprehension for prompt_list in prompt_with_auxiliaries_collate uses an incorrect iteration pattern. Instead of directly extracting the "text" field from each row, it iterates over all key-value pairs within each row, which is an overly complex and indirect way to get one text value per data item.