Is the default request latency correct?
Hello,
in the code we find (https://huggingface.co/spaces/genai-impact/ecologits-calculator/blob/main/app.py#L152-L157)
if models.find_model(provider, model) is not None:
impacts = compute_llm_impacts(
provider=provider,
model_name=model,
output_token_count=prompt,
request_latency=100000
)
with the doc string:
request_latency: Measured request latency in seconds.
100000 seconds seems far too long. Is it correct?
Hey Louis,
In EcoLogits, we estimate the latency with a linear regression, and we use the request_latency as an upper bound (as shown here : https://github.com/genai-impact/ecologits/blob/18b8840440b38c45a245f897bf5e33e3e05bd72f/ecologits/impacts/llm.py#L78). Putting 100000s just enforces the use of the regression, we could have put math.inf, or even None (following this line https://github.com/genai-impact/ecologits/blob/18b8840440b38c45a245f897bf5e33e3e05bd72f/ecologits/impacts/llm.py#L452)
I hope it answers your question!
Thanks for your answer, make sense.