Skip to content

LLM Workshop materials - update benchmarking and add fine-tuning

Terry Barrett requested to merge workshop into main

This is the first project in our new user-facing GitLab group. What is on main is what has been available to users for a while now (as a project with the same name in my "personal" UVM GitLab space). What I am looking to get reviewed here are the updates in support of the upcoming LLM workshop:

  • Use, by default, of the two new Python-runtime-environment modules created by @rminviel (llama-3-benchmarking-a100-env/0.1.0 and llama-3-finetuning-a100-env/0.1.0) so users will not need to create any conda environments to run the workshop materials on the A100 node reserved for the workshop.
  • Addition of an example for fine-tuning the model
  • Making the benchmarking more user friendly
  • Adding scripts for custom prompting
  • Instructions for setting up and running benchmarking and fine-tuning via the procedures detailed on the README
  • Support for more than one Llama job on a node at a time (via TMPDIR=$(mktemp -d) and picking a random port with torchrun option --rdzv-endpoint=localhost:0)

There are many files here, but most are .out files from cluster-wide Llama-3 benchmarking runs that don't need review. If you look at any of these, I recommend the latest run, Run14, which will be the ones I refer to in the workshop.

You have all been added to the workshop reservation so can run the workshop materials as-is.

I'd love at least one pair of other eyes on the documentation, mainly the README. I don't expect that attendees will get all the orientation they need from this documentation (there will be a slidedeck in the workshop itself), but hopefully those attendees who want to try things out before the workshop will be able to from what is shared/documented here.

When one person hits "Approve" I'll merge to main and share with the workshop attendees so those who are interested can try it out before the workshop and bring their questions. Review/updates can continue after this, I just owe the attendees something to start checking out.

I'll be happy to give an orientation to this project for anyone up for reviewing it.

Edited by Terry Barrett

Merge request reports