Carbon Dioxide Emissions

#1057
by danieljhand - opened

When I hover over the CO2 column on the leaderboard, I see a tooltip stating that the value only focuses on model inference for our specific setup. When I click on "Carbon Emissions Documentation" in the Learn More section, I get a 404 error. Can this be fixed?

Searching around, I found the following page https://huggingface.co/docs/hub/en/model-cards-co2 which suggests that the reported values of CO2 relate to training or fine-tuning of models. Can someone please confirm how I should read this column. Inference vs training/tuning? I suspect it's the latter but given that some models build upon the training of the base model, shouldn't the CO2 cost be cumulative? For example, Dracarys-72B-Instruct is a fine-tuned model based on Qwen2-72B-Instruct. Shouldn't it therefore have a greater CO2 cost? The table shows its cost to be approximately 1/3 of Qwen which is a little misleading. I'm keen to hear how others are using this data.

Many thanks in advance.

Open LLM Leaderboard org

Hi!
The documentation is here: https://huggingface.co/docs/leaderboards/open_llm_leaderboard/emissions :)
Can you provide a screenshot of where the link is broken?

Thank you @clefourrier . Please refer to the screenshot below. The link "Carbon Emissions Documentation" returns a 404.

Screenshot 2025-01-03 at 3.58.54 PM.png

Open LLM Leaderboard org

Thanks a lot, just fixed it!

clefourrier changed discussion status to closed

Sign up or log in to comment