README.md

---
tags:
- setfit
- sentence-transformers
- text-classification
- generated_from_setfit_trainer
widget:
- text: "\n\nHuman: 2. Cook the mixture for 5-7 minutes or until the berries have\
    \ softened and the sugar has dissolved, stirring occasionally.\n\nAssistant: \
    \ <sep> Sure, here are the steps to follow:\n\n1. Place your mixture on the stove\
    \ and turn the heat to medium.\n2. Start a timer for 5-7 minutes.\n3. As the mixture\
    \ cooks, make sure to stir it occasionally. This will prevent it from sticking\
    \ to the bottom of the pan and ensure that the heat is evenly distributed.\n4.\
    \ Keep an eye on the berries in the mixture. You'll know they've softened when\
    \ they can be easily squished with a spoon or spatula.\n5. Also, pay attention\
    \ to the sugar in the mixture. It should dissolve completely, meaning you won't\
    \ see any granules.\n6. Once the berries have softened and the sugar has dissolved,\
    \ your mixture is ready. \n\nRemember to be careful when handling the hot mixture.\
    \ Enjoy your cooking!"
- text: '


    Human: 25 short, practical English phrases for tour operator drivers who pick
    up clients at the airport. Each sentence should contain a maximum of five words.


    Assistant:  <sep> 1. "Welcome to our city!"

    2. "Need help with luggage?"

    3. "This way to the vehicle."

    4. "Please fasten your seatbelt."

    5. "We''re heading to your hotel."

    6. "Enjoy the complimentary water."

    7. "The ride will take 20 minutes."

    8. "Do you need a map?"

    9. "Restroom stop? Just let me know."

    10. "Here''s some local information."

    11. "Weather''s good for sightseeing."

    12. "Any dietary restrictions for meals?"

    13. "Tomorrow''s itinerary starts at 9."

    14. "Need recommendations for dinner?"

    15. "Hotel check-in is at 3."

    16. "Your tour guide is Jane."

    17. "Breakfast is served at 7."

    18. "Wifi password is ''welcome123''."

    19. "Any allergies I should know?"

    20. "Enjoy your stay here."

    21. "Your safety is our priority."

    22. "Excursion starts after breakfast."

    23. "Please keep your belongings safe."

    24. "Hope you had a good flight."

    25. "See you tomorrow morning."'
- text: '


    Human: 252 can be expressed as a product of primes as :

    Options:

    (A) 2 x 2 x 3 x 3 x 7

    (B) 2 x 2 x 2 x 3 x 7

    (C) 3 x 3 x 3 x 3 x 7

    (D) 2 x 3 x 3 x 3 x 7

    (E) 2 x 3 x 7 x 3 x 2


    252 = 2 x 2 x 3 x 3 x 7. So the answer is: (A).


    In a race with 30 runners where 5 trophies will be given to the top 7 runners
    (the trophies are distinct: first place, second place, etc), how many ways can
    this be done?

    Options:

    (A) 8^8 ways

    (B) 8^9 ways

    (C) 7^5 ways

    (D) 8^7 ways

    (E) 8^6 ways


    7 people can be prized with 5 distinct prizes in 7^5 ways So the answer is: (C).


    The edge of a cube is 6a cm. Find its surface?

    Options:

    (A) 216a2

    (B) 24a4

    (C) 24a1

    (D) 24a2

    (E) 24a7


    6a2 = 6 * 6a * 6a = 216a2 So the answer is: (A).


    In a school of 450 boys, 44% of Muslims, 28% Hindus, 10% Sikhs and the remaining
    of other communities. How many belonged to the other communities?

    Options:

    (A) 72

    (B) 75

    (C) 80

    (D) 81

    (E) 90


    Assistant:  <sep> Question: In a school of 450 boys, 44% of Muslims, 28% Hindus,
    10% Sikhs, and the remaining of other communities. How many belonged to the other
    communities?


    Answer: (E) 90


    Confidence Level: 90%


    Explanation: First, calculate the total number of boys in each religion:

    Christians: 100% - (44% + 28% + 10% + other) = 28%


    To find the number of Christians, multiply the total number of boys by the percentage
    of Christians:

    28% \* 450 = 125


    Now calculate the number of boys in other communities:

    5% + 2% = 7%

    7% \* 450 = 315


    Subtract the number of Christians from the total number of boys:

    450 - 125 = 325


    The number of boys in other communities is 325, but this includes Sikhs, which
    are not part of "other communities." So, to find the number of boys in other communities
    excluding Sikhs, subtract the number of Sikhs:

    325 - 10% \* 450 = 325 - 45 = 280


    So, the number of boys in other communities excluding Sikhs is 280, which corresponds
    to option (E).'
- text: '


    Human: 2. Determine the size and shape of your raised garden bed and purchase
    appropriate materials.


    Assistant:  <sep> The size and shape of your raised garden bed will depend on
    your available space and the type of plants you want to grow. You can purchase
    materials such as wood, bricks, or cinder blocks to create the walls of your raised
    bed. Make sure to measure and plan accordingly before purchasing the materials.'
- text: '


    Human: 27             : #define DEBUG_TYPE "globalisel-utils"

    53             :   // Assume physical registers are properly constrained.

    71             :            "Register class constraint is required unless either
    the "

    82             :     // and they never reach this function.

    101             :     // There''s nothing to be done on non-register operands.

    109             :     // Physical registers don''t need to be constrained.

    114             :     // to be constrained.

    119             :     // insert COPYs if that''s impossible.

    120             :     // constrainOperandRegClass does that for us.

    138             :   // a side-effect of some sort.

    143             :   // Instructions without side-effects are dead iff they only
    define dead vregs.

    162             :   // makes the diagnostic less useful) or if we''re going to
    emit a raw error.

    179             :   // Printing MI is expensive;  only do it if expensive remarks
    are enabled. Based on the passage above, What is the purpose of the DEBUG_TYPE
    macro defined in line 27?


    Assistant:  <sep> The purpose of the DEBUG_TYPE macro defined in line 27 is to
    serve as a custom logger in the GlobalISelUtils.cpp code. When instantiated, it
    sets a custom type identifier for logging messages during the compilation process.
    This helps to categorize and filter logging messages related to GlobalISelUtils
    in the compiler''s output, making it easier to track and diagnose issues specifically
    related to this utility.'
metrics:
- accuracy
pipeline_tag: text-classification
library_name: setfit
inference: false
base_model: sentence-transformers/all-mpnet-base-v2
model-index:
- name: SetFit with sentence-transformers/all-mpnet-base-v2
  results:
  - task:
      type: text-classification
      name: Text Classification
    dataset:
      name: Unknown
      type: unknown
      split: test
    metrics:
    - type: accuracy
      value: 0.7
      name: Accuracy
---

# SetFit with sentence-transformers/all-mpnet-base-v2

This is a [SetFit](https://github.com/huggingface/setfit) model that can be used for Text Classification. This SetFit model uses [sentence-transformers/all-mpnet-base-v2](https://huggingface.co/sentence-transformers/all-mpnet-base-v2) as the Sentence Transformer embedding model. A MultiOutputClassifier instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

1. Fine-tuning a [Sentence Transformer](https://www.sbert.net) with contrastive learning.
2. Training a classification head with features from the fine-tuned Sentence Transformer.

## Model Details

### Model Description
- **Model Type:** SetFit
- **Sentence Transformer body:** [sentence-transformers/all-mpnet-base-v2](https://huggingface.co/sentence-transformers/all-mpnet-base-v2)
- **Classification head:** a MultiOutputClassifier instance
- **Maximum Sequence Length:** 384 tokens
<!-- - **Number of Classes:** Unknown -->
<!-- - **Training Dataset:** [Unknown](https://huggingface.co/datasets/unknown) -->
<!-- - **Language:** Unknown -->
<!-- - **License:** Unknown -->

### Model Sources

- **Repository:** [SetFit on GitHub](https://github.com/huggingface/setfit)
- **Paper:** [Efficient Few-Shot Learning Without Prompts](https://arxiv.org/abs/2209.11055)
- **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)

## Evaluation

### Metrics
| Label   | Accuracy |
|:--------|:---------|
| **all** | 0.84     |

## Uses

### Direct Use for Inference

First install the SetFit library:

```bash
pip install setfit
```

Then you can load this model and run inference.

```python
from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("osmedi/LLM_response_evaluator")
# Run inference
preds = model("

Human: 2. Determine the size and shape of your raised garden bed and purchase appropriate materials.

Assistant:  <sep> The size and shape of your raised garden bed will depend on your available space and the type of plants you want to grow. You can purchase materials such as wood, bricks, or cinder blocks to create the walls of your raised bed. Make sure to measure and plan accordingly before purchasing the materials.")
```

<!--
### Downstream Use

*List how someone could finetune this model on their own dataset.*
-->

<!--
### Out-of-Scope Use

*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->

<!--
## Bias, Risks and Limitations

*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->

<!--
### Recommendations

*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->

## Training Details

### Training Set Metrics
| Training set | Min | Median   | Max   |
|:-------------|:----|:---------|:------|
| Word count   | 7   | 280.0747 | 15755 |

### Training Hyperparameters
- batch_size: (8, 8)
- num_epochs: (1, 1)
- max_steps: -1
- sampling_strategy: oversampling
- num_iterations: 2
- body_learning_rate: (2e-05, 2e-05)
- head_learning_rate: 2e-05
- loss: CosineSimilarityLoss
- distance_metric: cosine_distance
- margin: 0.25
- end_to_end: False
- use_amp: False
- warmup_proportion: 0.1
- l2_weight: 0.01
- seed: 42
- eval_max_steps: -1
- load_best_model_at_end: False

### Training Results
| Epoch  | Step | Training Loss | Validation Loss |
|:------:|:----:|:-------------:|:---------------:|
| 0.0011 | 1    | 0.593         | -               |
| 0.0526 | 50   | 0.3063        | -               |
| 0.1053 | 100  | 0.2582        | -               |
| 0.1579 | 150  | 0.2625        | -               |
| 0.2105 | 200  | 0.2477        | -               |
| 0.2632 | 250  | 0.2553        | -               |
| 0.3158 | 300  | 0.2473        | -               |
| 0.3684 | 350  | 0.2442        | -               |
| 0.4211 | 400  | 0.2368        | -               |
| 0.4737 | 450  | 0.2291        | -               |
| 0.5263 | 500  | 0.229         | -               |
| 0.5789 | 550  | 0.224         | -               |
| 0.6316 | 600  | 0.1974        | -               |
| 0.6842 | 650  | 0.2138        | -               |
| 0.7368 | 700  | 0.208         | -               |
| 0.7895 | 750  | 0.1936        | -               |
| 0.8421 | 800  | 0.2061        | -               |
| 0.8947 | 850  | 0.1931        | -               |
| 0.9474 | 900  | 0.1868        | -               |
| 1.0    | 950  | 0.186         | -               |

### Framework Versions
- Python: 3.10.12
- SetFit: 1.1.0
- Sentence Transformers: 3.3.1
- Transformers: 4.44.2
- PyTorch: 2.5.1+cu121
- Datasets: 3.2.0
- Tokenizers: 0.19.1

## Citation

### BibTeX
```bibtex
@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}
```

<!--
## Glossary

*Clearly define terms in order to be accessible across audiences.*
-->

<!--
## Model Card Authors

*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->

<!--
## Model Card Contact

*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->