Add SetFit model

README.md

@@ -16,11 +16,11 @@ tags:
 - text-classification
 - generated_from_setfit_trainer
 widget:
-- text: Of what discipline is affective computing a branch?
-- text: why is my mitsubishi aircon light blinking
+- text: the nerves,blood vessels, and glands are located in which layer of the skin
+- text: Where would you put refuse if you do not want it to exist any more?
 - text: Obesity can cause resistance to which hormone?
-- text: farm beer garden - ohio
-- text: Where did Reagan and Gorbachev have their Star Wars summit in October 19865?
+- text: Referees
+- text: where does the water at niagra falls come from
 inference: true
 model-index:
 - name: SetFit with BAAI/bge-small-en-v1.5 on Health Information Needs
@@ -35,33 +35,33 @@ model-index:
     metrics:
     - type: '0'
       value:
-        precision: 0.5862573099415205
-        recall: 0.9796416938110749
-        f1-score: 0.7335365853658536
+        precision: 0.36527485731450887
+        recall: 0.990228013029316
+        f1-score: 0.5336844415185429
         support: 1228.0
       name: '0'
     - type: '1'
       value:
-        precision: 0.9926318891836133
-        recall: 0.7986720417358312
-        f1-score: 0.8851511169513797
+        precision: 0.994328922495274
+        recall: 0.4989328906805786
+        f1-score: 0.6644560240012632
         support: 4217.0
       name: '1'
     - type: accuracy
-      value: 0.8394857667584941
+      value: 0.6097337006427915
       name: Accuracy
     - type: macro avg
       value:
-        precision: 0.789444599562567
-        recall: 0.889156867773453
-        f1-score: 0.8093438511586166
+        precision: 0.6798018899048914
+        recall: 0.7445804518549473
+        f1-score: 0.5990702327599031
         support: 5445.0
       name: Macro Avg
     - type: weighted avg
       value:
-        precision: 0.9009830400909982
-        recall: 0.8394857667584941
-        f1-score: 0.8509577937581702
+        precision: 0.8524596126620364
+        recall: 0.6097337006427915
+        f1-score: 0.6349633695864275
         support: 5445.0
       name: Weighted Avg
 ---
@@ -94,17 +94,17 @@ The model has been trained using an efficient few-shot learning technique that i
 - **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
 
 ### Model Labels
-| Label | Examples                                                                                                                                                    |
-|:------|:------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| 1     | <ul><li>'why is my mitsubishi aircon light blinking'</li><li>'what was legalism'</li><li>'farm beer garden - ohio'</li></ul>                                |
-| 0     | <ul><li>'what makes an adult vulnerable'</li><li>'Of what discipline is affective computing a branch?'</li><li>'What does a ribosome consist of?'</li></ul> |
+| Label | Examples                                                                                                                                                                                                                |
+|:------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| 1     | <ul><li>'Where would you put refuse if you do not want it to exist any more?'</li><li>'Which is an example of the absolutism under peter the great?'</li><li>'where does the water at niagra falls come from'</li></ul> |
+| 0     | <ul><li>'the nerves,blood vessels, and glands are located in which layer of the skin'</li><li>'Of what discipline is affective computing a branch?'</li><li>'Obesity can cause resistance to which hormone?'</li></ul>  |
 
 ## Evaluation
 
 ### Metrics
-| Label   | 0                                                                                                                  | 1                                                                                                                  | Accuracy | Macro Avg                                                                                                        | Weighted Avg                                                                                                       |
-|:--------|:-------------------------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------|:---------|:-----------------------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------|
-| **all** | {'precision': 0.5862573099415205, 'recall': 0.9796416938110749, 'f1-score': 0.7335365853658536, 'support': 1228.0} | {'precision': 0.9926318891836133, 'recall': 0.7986720417358312, 'f1-score': 0.8851511169513797, 'support': 4217.0} | 0.8395   | {'precision': 0.789444599562567, 'recall': 0.889156867773453, 'f1-score': 0.8093438511586166, 'support': 5445.0} | {'precision': 0.9009830400909982, 'recall': 0.8394857667584941, 'f1-score': 0.8509577937581702, 'support': 5445.0} |
+| Label   | 0                                                                                                                  | 1                                                                                                                 | Accuracy | Macro Avg                                                                                                          | Weighted Avg                                                                                                       |
+|:--------|:-------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------|:---------|:-------------------------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------|
+| **all** | {'precision': 0.36527485731450887, 'recall': 0.990228013029316, 'f1-score': 0.5336844415185429, 'support': 1228.0} | {'precision': 0.994328922495274, 'recall': 0.4989328906805786, 'f1-score': 0.6644560240012632, 'support': 4217.0} | 0.6097   | {'precision': 0.6798018899048914, 'recall': 0.7445804518549473, 'f1-score': 0.5990702327599031, 'support': 5445.0} | {'precision': 0.8524596126620364, 'recall': 0.6097337006427915, 'f1-score': 0.6349633695864275, 'support': 5445.0} |
 
 ## Uses
 
@@ -124,7 +124,7 @@ from setfit import SetFitModel
 # Download from the 🤗 Hub
 model = SetFitModel.from_pretrained("setfit_model_id")
 # Run inference
-preds = model("farm beer garden - ohio")
+preds = model("Referees")
 ```
 
 <!--
@@ -156,12 +156,12 @@ preds = model("farm beer garden - ohio")
 ### Training Set Metrics
 | Training set | Min | Median | Max |
 |:-------------|:----|:-------|:----|
-| Word count   | 1   | 7.2    | 15  |
+| Word count   | 1   | 7.3    | 15  |
 
 | Label | Training Sample Count |
 |:------|:----------------------|
-| 0     | 10                    |
-| 1     | 10                    |
+| 0     | 5                     |
+| 1     | 5                     |
 
 ### Training Hyperparameters
 - batch_size: (32, 32)
@@ -182,10 +182,9 @@ preds = model("farm beer garden - ohio")
 - load_best_model_at_end: False
 
 ### Training Results
-| Epoch  | Step | Training Loss | Validation Loss |
-|:------:|:----:|:-------------:|:---------------:|
-| 0.1429 | 1    | 0.1987        | -               |
-| 7.1429 | 50   | 0.1561        | -               |
+| Epoch | Step | Training Loss | Validation Loss |
+|:-----:|:----:|:-------------:|:---------------:|
+| 0.5   | 1    | 0.1884        | -               |
 
 ### Framework Versions
 - Python: 3.12.2

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:f52891a11de3ef11691349f96f962708418936b143a844da3340264289a361f7
+oid sha256:486ea20f55ffb50896c78a545f8ca61471c6d8a206cfa10f5fdfe3af6eb95c79
 size 133462128

model_head.pkl

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:e096f2ae7b7a0003dbde2fcd0662ef293d263c69ec1f4bed6ecffd3c1907a102
+oid sha256:8f9cc29255d2c9868230092de6a5de0a30183110524ea434529c78c21306ea6f
 size 3935