|
--- |
|
license: mit |
|
datasets: |
|
- sms1097/self_rag_tokens_train_data |
|
--- |
|
# Utility Model |
|
|
|
This generates the `IsUseful` token as descirbed in Self-RAG. |
|
|
|
We are testing to see if an answer is useful to the given user question. We output a score from 1-5 based on how useful the answer is. |
|
|
|
The expected input to the model is: |
|
|
|
``` |
|
Instruction: {instruction}\nAnswer: {answer}", |
|
``` |
|
|
|
|
|
### Training Results |
|
``` |
|
{'eval_loss': 0.4719298779964447, |
|
'eval_mse': 0.4719298183917999, |
|
'eval_mae': 0.25655537843704224, |
|
'eval_r2': 0.5200293292355334, |
|
'eval_accuracy': 0.9001516683518705} |
|
``` |
