docs: update LLM tutorial to optimize and evaluate large language models

[davidberenstein1957]

• Added a new section for optimizing and evaluating large language models using the SmashConfig object.
• Updated the tutorial to include installation instructions, model loading, and inference steps.
• Introduced evaluation metrics such as elapsed time and perplexity for assessing model performance.
• Removed outdated references and improved clarity throughout the tutorial.

Description

Related Issue

Fixes #121

Type of Change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

How Has This Been Tested?

Checklist

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes

Additional Notes

[sharpenb]

Thanks for the tutorial! I have mostly similar comments as in the image generation PR #127 (e.g. tutorial card, section org....). Beyond these other comments, here some other:

• We could add some other efficiency emtrics inlcuding memeory, latency (like in the AI courses)
• We should explain why we use a small llm and explain that it works with others.

[sharpenb]

Thanks for the update :) I have mostly some shared comments with the image generatino PR. Some further comments:

• Could we add the minimal config for the device also for the iamge generation tutorial?
• Could we add memory and througput metrics? These are important when quantizing LLMs.
• It is not clear that we need inference_args to be udpated

[davidberenstein1957]

@gtregoat @sharpenb @SaboniAmine could you review this PR, I think it would be nice to merge before the release.

[sharpenb]

Thanks for the update! A last update would be to indicate the units with througput (samples/s), latency (s/samples) and Base Model, Compressed Model, Relative Difference in the row/columns names to be complete. Could actually also add CO2 emissions which is in the evaluation agent. COuld we also have rather token/second, second/token, and ideallytime to first token? :)

[davidberenstein1957]

Thanks for the update! A last update would be to indicate the units with througput (samples/s), latency (s/samples) and Base Model, Compressed Model, Relative Difference in the row/columns names to be complete. Could actually also add CO2 emissions which is in the evaluation agent. COuld we also have rather token/second, second/token, and ideallytime to first token? :)

Hi @sharpenb, I updated the metrics so that I don't require me to boot up a VM and rerun the notebook another time. Hope that is enough for now, for the ASR or video tutorial we can add some additional merics.

[sharpenb]

Thanks for the update! The tutorial is very nice. I would be super nice if we can add the time-to-first-token adn token time instead of total time but this should be good for now.

[nifleisch]

Cool tutorial with impressive results! I found a few minor errors that you might want to fix before merging:

• "Additionally, we lower the n_iterations and n_warmup_iterations to ensure that we monitor the performance of the model whenever it is running smoothly." I think while this explains well why we would like to use warmup steps in general, it does not justify why we reduce n_warmup_iterations here.
• In the text you mention that we limit the dataset like this: datamodule.limit_datasets(10), but in the code block below we do datamodule.limit_datasets(100)
• Similarly, in the text it is mentioned that we use hqq_weight_bits=4 while we are using hqq_weight_bits=8 in the code

[sharpenb]

We need to do a last fix ;)

asked Sara to do the final approval as you are not logged in