metadata
tags:
- setfit
- sentence-transformers
- text-classification
- generated_from_setfit_trainer
widget:
- text: >-
Human: 2. Cook the mixture for 5-7 minutes or until the berries have
softened and the sugar has dissolved, stirring occasionally.
Assistant: <sep> Sure, here are the steps to follow:
1. Place your mixture on the stove and turn the heat to medium.
2. Start a timer for 5-7 minutes.
3. As the mixture cooks, make sure to stir it occasionally. This will
prevent it from sticking to the bottom of the pan and ensure that the heat
is evenly distributed.
4. Keep an eye on the berries in the mixture. You'll know they've softened
when they can be easily squished with a spoon or spatula.
5. Also, pay attention to the sugar in the mixture. It should dissolve
completely, meaning you won't see any granules.
6. Once the berries have softened and the sugar has dissolved, your
mixture is ready.
Remember to be careful when handling the hot mixture. Enjoy your cooking!
- text: >-
Human: 25 short, practical English phrases for tour operator drivers who
pick up clients at the airport. Each sentence should contain a maximum of
five words.
Assistant: <sep> 1. "Welcome to our city!"
2. "Need help with luggage?"
3. "This way to the vehicle."
4. "Please fasten your seatbelt."
5. "We're heading to your hotel."
6. "Enjoy the complimentary water."
7. "The ride will take 20 minutes."
8. "Do you need a map?"
9. "Restroom stop? Just let me know."
10. "Here's some local information."
11. "Weather's good for sightseeing."
12. "Any dietary restrictions for meals?"
13. "Tomorrow's itinerary starts at 9."
14. "Need recommendations for dinner?"
15. "Hotel check-in is at 3."
16. "Your tour guide is Jane."
17. "Breakfast is served at 7."
18. "Wifi password is 'welcome123'."
19. "Any allergies I should know?"
20. "Enjoy your stay here."
21. "Your safety is our priority."
22. "Excursion starts after breakfast."
23. "Please keep your belongings safe."
24. "Hope you had a good flight."
25. "See you tomorrow morning."
- text: >-
Human: 252 can be expressed as a product of primes as :
Options:
(A) 2 x 2 x 3 x 3 x 7
(B) 2 x 2 x 2 x 3 x 7
(C) 3 x 3 x 3 x 3 x 7
(D) 2 x 3 x 3 x 3 x 7
(E) 2 x 3 x 7 x 3 x 2
252 = 2 x 2 x 3 x 3 x 7. So the answer is: (A).
In a race with 30 runners where 5 trophies will be given to the top 7
runners (the trophies are distinct: first place, second place, etc), how
many ways can this be done?
Options:
(A) 8^8 ways
(B) 8^9 ways
(C) 7^5 ways
(D) 8^7 ways
(E) 8^6 ways
7 people can be prized with 5 distinct prizes in 7^5 ways So the answer
is: (C).
The edge of a cube is 6a cm. Find its surface?
Options:
(A) 216a2
(B) 24a4
(C) 24a1
(D) 24a2
(E) 24a7
6a2 = 6 * 6a * 6a = 216a2 So the answer is: (A).
In a school of 450 boys, 44% of Muslims, 28% Hindus, 10% Sikhs and the
remaining of other communities. How many belonged to the other
communities?
Options:
(A) 72
(B) 75
(C) 80
(D) 81
(E) 90
Assistant: <sep> Question: In a school of 450 boys, 44% of Muslims, 28%
Hindus, 10% Sikhs, and the remaining of other communities. How many
belonged to the other communities?
Answer: (E) 90
Confidence Level: 90%
Explanation: First, calculate the total number of boys in each religion:
Christians: 100% - (44% + 28% + 10% + other) = 28%
To find the number of Christians, multiply the total number of boys by the
percentage of Christians:
28% \* 450 = 125
Now calculate the number of boys in other communities:
5% + 2% = 7%
7% \* 450 = 315
Subtract the number of Christians from the total number of boys:
450 - 125 = 325
The number of boys in other communities is 325, but this includes Sikhs,
which are not part of "other communities." So, to find the number of boys
in other communities excluding Sikhs, subtract the number of Sikhs:
325 - 10% \* 450 = 325 - 45 = 280
So, the number of boys in other communities excluding Sikhs is 280, which
corresponds to option (E).
- text: >-
Human: 2. Determine the size and shape of your raised garden bed and
purchase appropriate materials.
Assistant: <sep> The size and shape of your raised garden bed will depend
on your available space and the type of plants you want to grow. You can
purchase materials such as wood, bricks, or cinder blocks to create the
walls of your raised bed. Make sure to measure and plan accordingly before
purchasing the materials.
- text: >-
Human: 27 : #define DEBUG_TYPE "globalisel-utils"
53 : // Assume physical registers are properly constrained.
71 : "Register class constraint is required unless
either the "
82 : // and they never reach this function.
101 : // There's nothing to be done on non-register
operands.
109 : // Physical registers don't need to be constrained.
114 : // to be constrained.
119 : // insert COPYs if that's impossible.
120 : // constrainOperandRegClass does that for us.
138 : // a side-effect of some sort.
143 : // Instructions without side-effects are dead iff they
only define dead vregs.
162 : // makes the diagnostic less useful) or if we're going
to emit a raw error.
179 : // Printing MI is expensive; only do it if expensive
remarks are enabled. Based on the passage above, What is the purpose of
the DEBUG_TYPE macro defined in line 27?
Assistant: <sep> The purpose of the DEBUG_TYPE macro defined in line 27
is to serve as a custom logger in the GlobalISelUtils.cpp code. When
instantiated, it sets a custom type identifier for logging messages during
the compilation process. This helps to categorize and filter logging
messages related to GlobalISelUtils in the compiler's output, making it
easier to track and diagnose issues specifically related to this utility.
metrics:
- accuracy
pipeline_tag: text-classification
library_name: setfit
inference: false
base_model: sentence-transformers/all-mpnet-base-v2
model-index:
- name: SetFit with sentence-transformers/all-mpnet-base-v2
results:
- task:
type: text-classification
name: Text Classification
dataset:
name: Unknown
type: unknown
split: test
metrics:
- type: accuracy
value: 0.7
name: Accuracy
SetFit with sentence-transformers/all-mpnet-base-v2
This is a SetFit model that can be used for Text Classification. This SetFit model uses sentence-transformers/all-mpnet-base-v2 as the Sentence Transformer embedding model. A MultiOutputClassifier instance is used for classification.
The model has been trained using an efficient few-shot learning technique that involves:
- Fine-tuning a Sentence Transformer with contrastive learning.
- Training a classification head with features from the fine-tuned Sentence Transformer.
Model Details
Model Description
- Model Type: SetFit
- Sentence Transformer body: sentence-transformers/all-mpnet-base-v2
- Classification head: a MultiOutputClassifier instance
- Maximum Sequence Length: 384 tokens
Model Sources
- Repository: SetFit on GitHub
- Paper: Efficient Few-Shot Learning Without Prompts
- Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts
Evaluation
Metrics
| Label | Accuracy |
|---|---|
| all | 0.84 |
Uses
Direct Use for Inference
First install the SetFit library:
pip install setfit
Then you can load this model and run inference.
from setfit import SetFitModel
# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("osmedi/LLM_response_evaluator")
# Run inference
preds = model("
Human: 2. Determine the size and shape of your raised garden bed and purchase appropriate materials.
Assistant: <sep> The size and shape of your raised garden bed will depend on your available space and the type of plants you want to grow. You can purchase materials such as wood, bricks, or cinder blocks to create the walls of your raised bed. Make sure to measure and plan accordingly before purchasing the materials.")
Training Details
Training Set Metrics
| Training set | Min | Median | Max |
|---|---|---|---|
| Word count | 7 | 280.0747 | 15755 |
Training Hyperparameters
- batch_size: (8, 8)
- num_epochs: (1, 1)
- max_steps: -1
- sampling_strategy: oversampling
- num_iterations: 2
- body_learning_rate: (2e-05, 2e-05)
- head_learning_rate: 2e-05
- loss: CosineSimilarityLoss
- distance_metric: cosine_distance
- margin: 0.25
- end_to_end: False
- use_amp: False
- warmup_proportion: 0.1
- l2_weight: 0.01
- seed: 42
- eval_max_steps: -1
- load_best_model_at_end: False
Training Results
| Epoch | Step | Training Loss | Validation Loss |
|---|---|---|---|
| 0.0011 | 1 | 0.593 | - |
| 0.0526 | 50 | 0.3063 | - |
| 0.1053 | 100 | 0.2582 | - |
| 0.1579 | 150 | 0.2625 | - |
| 0.2105 | 200 | 0.2477 | - |
| 0.2632 | 250 | 0.2553 | - |
| 0.3158 | 300 | 0.2473 | - |
| 0.3684 | 350 | 0.2442 | - |
| 0.4211 | 400 | 0.2368 | - |
| 0.4737 | 450 | 0.2291 | - |
| 0.5263 | 500 | 0.229 | - |
| 0.5789 | 550 | 0.224 | - |
| 0.6316 | 600 | 0.1974 | - |
| 0.6842 | 650 | 0.2138 | - |
| 0.7368 | 700 | 0.208 | - |
| 0.7895 | 750 | 0.1936 | - |
| 0.8421 | 800 | 0.2061 | - |
| 0.8947 | 850 | 0.1931 | - |
| 0.9474 | 900 | 0.1868 | - |
| 1.0 | 950 | 0.186 | - |
Framework Versions
- Python: 3.10.12
- SetFit: 1.1.0
- Sentence Transformers: 3.3.1
- Transformers: 4.44.2
- PyTorch: 2.5.1+cu121
- Datasets: 3.2.0
- Tokenizers: 0.19.1
Citation
BibTeX
@article{https://doi.org/10.48550/arxiv.2209.11055,
doi = {10.48550/ARXIV.2209.11055},
url = {https://arxiv.org/abs/2209.11055},
author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
title = {Efficient Few-Shot Learning Without Prompts},
publisher = {arXiv},
year = {2022},
copyright = {Creative Commons Attribution 4.0 International}
}