jangedoo commited on
Commit
ac7d4bd
·
verified ·
1 Parent(s): 9439fb2

Add new SentenceTransformer model.

Browse files
1_Pooling/config.json ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "word_embedding_dimension": 384,
3
+ "pooling_mode_cls_token": false,
4
+ "pooling_mode_mean_tokens": true,
5
+ "pooling_mode_max_tokens": false,
6
+ "pooling_mode_mean_sqrt_len_tokens": false,
7
+ "pooling_mode_weightedmean_tokens": false,
8
+ "pooling_mode_lasttoken": false,
9
+ "include_prompt": true
10
+ }
README.md ADDED
@@ -0,0 +1,493 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: sentence-transformers/all-MiniLM-L6-v2
3
+ datasets:
4
+ - momo22/eng2nep
5
+ language:
6
+ - en
7
+ - ne
8
+ library_name: sentence-transformers
9
+ metrics:
10
+ - negative_mse
11
+ - src2trg_accuracy
12
+ - trg2src_accuracy
13
+ - mean_accuracy
14
+ pipeline_tag: sentence-similarity
15
+ tags:
16
+ - sentence-transformers
17
+ - sentence-similarity
18
+ - feature-extraction
19
+ - generated_from_trainer
20
+ - dataset_size:1000
21
+ - loss:MSELoss
22
+ - dataset_size:5000
23
+ - dataset_size:8000
24
+ widget:
25
+ - source_sentence: 'The aggressive semi-employed religion workshop of Razzak, (EFP).
26
+
27
+ '
28
+ sentences:
29
+ - 'मा ग्रिटर भेट्टाउन सकेन वा GDM प्रयोगकर्ताले कार्यान्वयन गर्न सकेन
30
+
31
+ '
32
+ - 'रज्जाकको आक्रामक अर्द्धशतक धर्मशाला, (एएफपी)।
33
+
34
+ '
35
+ - 'त्यसैले मेरो विजयपछि म त्यस्तो अवस्था आउन दिनेछैन।
36
+
37
+ '
38
+ - source_sentence: 'The authority is being a constitutional body, it was also empowered
39
+ by passing the bill from Parliament.
40
+
41
+ '
42
+ sentences:
43
+ - 'अख्तियार संवैधानिक निकाय त हुँदै हो, त्यसमा पनि संसदबाटै विधेयक पास गरेर अख्तियारलाई
44
+ अधिकारसम्पन्न पनि गराइएको थियो।
45
+
46
+ '
47
+ - 'म यहूदाका राजा सिदकियाहलाई र उसका मानिसहरूलाई तिनीहरूका शत्रुहरूकहाँ सुम्पिनेछु
48
+ जसले तिनीहरूलाई मार्न चाहन्छन्। ती सेनाहरू यरूशलेमबाट गइसकेका भएता पनि म तिनीहरूलाई
49
+ बाबेलका राजाको सेनाहरूकहाँ सुम्पिनेछु।
50
+
51
+ '
52
+ - '– संकटकालको असर न्यायिक क्षेत्रमा मात्रै पर्दैन, समग्र मुलुकमै पर्छ।
53
+
54
+ '
55
+ - source_sentence: 'The two-day conference will participate in investors from China,
56
+ India, Japan, the US, European countries, Britain and other countries, the Federation
57
+ said.
58
+
59
+ '
60
+ sentences:
61
+ - 'उनीहरूको जनजीविकाको आधार प्राकृतिक स्रोत रहेको छ।
62
+
63
+ '
64
+ - 'दुई दिनसम्म हुने सम्मेलनमा चीन, भारत, जापान, अमेरिका, युरोपियन देशहरू, बेलायत
65
+ लगायत देशबाट लगानीकर्ताको सहभागिता गराउने महासंघले जानकारी दिएको छ
66
+
67
+ '
68
+ - 'नयाँ स्न्यापसट लिनका लागि यो बटन क्लिक गर्नुहोस् ।
69
+
70
+ '
71
+ - source_sentence: 'Mr Sankey issued a "confession" through his solicitor after Shields
72
+ had been convicted but then withdrew it.
73
+
74
+ '
75
+ sentences:
76
+ - 'श्री सान्कीले ढालहरू दोषी भएपछि आफ्नो समाधानकर्तामार्फत "स्वीकृति" जारी गर्नुभयो
77
+ तर त्यसपछि यसलाई फिर्ता लिनुभयो।
78
+
79
+ '
80
+ - 'कृत्रिम रुपमा पेट्रोलियम पदार्थको मूल्य स्थिर राख्न अनुदान दिदै जाने हो भने नेपाली
81
+ अर्थतन्त्र एकदिन धराशायी हुनेछ।
82
+
83
+ '
84
+ - 'ओली सरकारले "राष्ट्रियता-राष्ट्रवाद र" आर्थिक सम्ब्रिद्धि "-आर्थिक विकासलाई यसको
85
+ प्राथमिकताको रूपमा घोषणा गरेको छ।
86
+
87
+ '
88
+ - source_sentence: 'We want to use this time to appeal to the American government
89
+ to see if they can finally close this chapter.
90
+
91
+ '
92
+ sentences:
93
+ - 'धेरैले घाउ पाए र ओछ्यानमा थिए।
94
+
95
+ '
96
+ - 'नाम यसको अन्तरराष्ट्रिय हलको अद्वितिय डिजाइनबाट स्पष्ट रूपमा प्राप्त हुन्छ, जुन
97
+ शीर्षकनियम स्पेसबाट बनेको छ, जुन ठूलो गहिराइमा उच्च दबाब बुझ्न सक्षम छ।
98
+
99
+ '
100
+ - 'हामी अमेरिकी सरकारलाई अपील गर्न यसपटक प्रयोग गर्न चाहन्छौं कि उनीहरूले अन्त्यमा
101
+ यो अध्याय बन्द गर्न सक्छन्।
102
+
103
+ '
104
+ model-index:
105
+ - name: SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v2
106
+ results:
107
+ - task:
108
+ type: knowledge-distillation
109
+ name: Knowledge Distillation
110
+ dataset:
111
+ name: Unknown
112
+ type: unknown
113
+ metrics:
114
+ - type: negative_mse
115
+ value: -0.37439612206071615
116
+ name: Negative Mse
117
+ - task:
118
+ type: translation
119
+ name: Translation
120
+ dataset:
121
+ name: Unknown
122
+ type: unknown
123
+ metrics:
124
+ - type: src2trg_accuracy
125
+ value: 0.0186
126
+ name: Src2Trg Accuracy
127
+ - type: trg2src_accuracy
128
+ value: 0.00835
129
+ name: Trg2Src Accuracy
130
+ - type: mean_accuracy
131
+ value: 0.013474999999999999
132
+ name: Mean Accuracy
133
+ ---
134
+
135
+ # SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v2
136
+
137
+ This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [sentence-transformers/all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) on the [momo22/eng2nep](https://huggingface.co/datasets/momo22/eng2nep) dataset. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
138
+
139
+ ## Model Details
140
+
141
+ ### Model Description
142
+ - **Model Type:** Sentence Transformer
143
+ - **Base model:** [sentence-transformers/all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) <!-- at revision 8b3219a92973c328a8e22fadcfa821b5dc75636a -->
144
+ - **Maximum Sequence Length:** 256 tokens
145
+ - **Output Dimensionality:** 384 tokens
146
+ - **Similarity Function:** Cosine Similarity
147
+ - **Training Dataset:**
148
+ - [momo22/eng2nep](https://huggingface.co/datasets/momo22/eng2nep)
149
+ - **Languages:** en, ne
150
+ <!-- - **License:** Unknown -->
151
+
152
+ ### Model Sources
153
+
154
+ - **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
155
+ - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
156
+ - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
157
+
158
+ ### Full Model Architecture
159
+
160
+ ```
161
+ SentenceTransformer(
162
+ (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel
163
+ (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
164
+ (2): Normalize()
165
+ )
166
+ ```
167
+
168
+ ## Usage
169
+
170
+ ### Direct Usage (Sentence Transformers)
171
+
172
+ First install the Sentence Transformers library:
173
+
174
+ ```bash
175
+ pip install -U sentence-transformers
176
+ ```
177
+
178
+ Then you can load this model and run inference.
179
+ ```python
180
+ from sentence_transformers import SentenceTransformer
181
+
182
+ # Download from the 🤗 Hub
183
+ model = SentenceTransformer("jangedoo/all-MiniLM-L6-v2-nepali")
184
+ # Run inference
185
+ sentences = [
186
+ 'We want to use this time to appeal to the American government to see if they can finally close this chapter.\n',
187
+ 'हामी अमेरिकी सरकारलाई अपील गर्न यसपटक प्रयोग गर्न चाहन्छौं कि उनीहरूले अन्त्यमा यो अध्याय बन्द गर्न सक्छन्।\n',
188
+ 'नाम यसको अन्तरराष्ट्रिय हलको अद्वितिय डिजाइनबाट स्पष्ट रूपमा प्राप्त हुन्छ, जुन शीर्षकनियम स्पेसबाट बनेको छ, जुन ठूलो गहिराइमा उच्च दबाब बुझ्न सक्षम छ।\n',
189
+ ]
190
+ embeddings = model.encode(sentences)
191
+ print(embeddings.shape)
192
+ # [3, 384]
193
+
194
+ # Get the similarity scores for the embeddings
195
+ similarities = model.similarity(embeddings, embeddings)
196
+ print(similarities.shape)
197
+ # [3, 3]
198
+ ```
199
+
200
+ <!--
201
+ ### Direct Usage (Transformers)
202
+
203
+ <details><summary>Click to see the direct usage in Transformers</summary>
204
+
205
+ </details>
206
+ -->
207
+
208
+ <!--
209
+ ### Downstream Usage (Sentence Transformers)
210
+
211
+ You can finetune this model on your own dataset.
212
+
213
+ <details><summary>Click to expand</summary>
214
+
215
+ </details>
216
+ -->
217
+
218
+ <!--
219
+ ### Out-of-Scope Use
220
+
221
+ *List how the model may foreseeably be misused and address what users ought not to do with the model.*
222
+ -->
223
+
224
+ ## Evaluation
225
+
226
+ ### Metrics
227
+
228
+ #### Knowledge Distillation
229
+
230
+ * Evaluated with [<code>MSEEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.MSEEvaluator)
231
+
232
+ | Metric | Value |
233
+ |:-----------------|:------------|
234
+ | **negative_mse** | **-0.3744** |
235
+
236
+ #### Translation
237
+
238
+ * Evaluated with [<code>TranslationEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.TranslationEvaluator)
239
+
240
+ | Metric | Value |
241
+ |:------------------|:-----------|
242
+ | src2trg_accuracy | 0.0186 |
243
+ | trg2src_accuracy | 0.0083 |
244
+ | **mean_accuracy** | **0.0135** |
245
+
246
+ <!--
247
+ ## Bias, Risks and Limitations
248
+
249
+ *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
250
+ -->
251
+
252
+ <!--
253
+ ### Recommendations
254
+
255
+ *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
256
+ -->
257
+
258
+ ## Training Details
259
+
260
+ ### Training Dataset
261
+
262
+ #### momo22/eng2nep
263
+
264
+ * Dataset: [momo22/eng2nep](https://huggingface.co/datasets/momo22/eng2nep) at [57da8d4](https://huggingface.co/datasets/momo22/eng2nep/tree/57da8d44266896e334c1d8f2528cbbf666fbd0ca)
265
+ * Size: 8,000 training samples
266
+ * Columns: <code>English</code>, <code>Nepali</code>, and <code>label</code>
267
+ * Approximate statistics based on the first 1000 samples:
268
+ | | English | Nepali | label |
269
+ |:--------|:-----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|:-------------------------------------|
270
+ | type | string | string | list |
271
+ | details | <ul><li>min: 3 tokens</li><li>mean: 26.29 tokens</li><li>max: 130 tokens</li></ul> | <ul><li>min: 3 tokens</li><li>mean: 65.39 tokens</li><li>max: 256 tokens</li></ul> | <ul><li>size: 384 elements</li></ul> |
272
+ * Samples:
273
+ | English | Nepali | label |
274
+ |:------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------|
275
+ | <code>But with the origin of feudal practices in the Middle Ages, the practice of untouchability began, as well as discrimination against women.<br></code> | <code>तर मध्ययुगमा सामन्ती प्रथाको उद्भव भएसँगै जसरी छुवाछुत प्रथाको शुरुवात भयो, त्यसैगरी नारी प्रति पनि विभेद गरिन थालियो<br></code> | <code>[-0.05432726442813873, 0.029996933415532112, -0.008532932959496975, -0.035200122743844986, 0.008856767788529396, ...]</code> |
276
+ | <code>A Pandit was found on the way to Pokhara from Baglung.<br></code> | <code>वाग्लुङ्गबाट पोखरा आउँदा बाटोमा एकजना पण्डित भेटिए।<br></code> | <code>[-0.023763148114085197, 0.0959007516503334, -0.11197677254676819, 0.10978179425001144, -0.028137238696217537, ...]</code> |
277
+ | <code>He went on: "She ate a perfectly normal and healthy diet.<br></code> | <code>उनी गए: "उनले पूर्ण सामान्य र स्वस्थ आहार खाइन्।<br></code> | <code>[0.028130479156970978, 0.030386686325073242, -0.012276170775294304, 0.1316223442554474, -0.01928202621638775, ...]</code> |
278
+ * Loss: [<code>MSELoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#mseloss)
279
+
280
+ ### Evaluation Dataset
281
+
282
+ #### momo22/eng2nep
283
+
284
+ * Dataset: [momo22/eng2nep](https://huggingface.co/datasets/momo22/eng2nep) at [57da8d4](https://huggingface.co/datasets/momo22/eng2nep/tree/57da8d44266896e334c1d8f2528cbbf666fbd0ca)
285
+ * Size: 500 evaluation samples
286
+ * Columns: <code>English</code>, <code>Nepali</code>, and <code>label</code>
287
+ * Approximate statistics based on the first 1000 samples:
288
+ | | English | Nepali | label |
289
+ |:--------|:-----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:-------------------------------------|
290
+ | type | string | string | list |
291
+ | details | <ul><li>min: 4 tokens</li><li>mean: 26.71 tokens</li><li>max: 213 tokens</li></ul> | <ul><li>min: 3 tokens</li><li>mean: 64.1 tokens</li><li>max: 256 tokens</li></ul> | <ul><li>size: 384 elements</li></ul> |
292
+ * Samples:
293
+ | English | Nepali | label |
294
+ |:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------|
295
+ | <code>Chapter 3<br></code> | <code>परिच्छेद–३<br></code> | <code>[-0.049459926784038544, 0.048675183206796646, 0.016583453863859177, 0.04876156523823738, -0.020754676312208176, ...]</code> |
296
+ | <code>The capability of MOF would be strengthened to enable it to efficiently play the lead role in donor coordination, and to secure support from all stakeholders in aid coordination activities.<br></code> | <code>दाताहरूको समन्वयमा नेतृत्वदायीको भूमिका निर्वाह प्रभावकारी ढंगले गर्न अर्थ मन्त्रालयको क्षमता सुदृढ गरिनेछ यसको लागि सबै सरोकारवालाबाट समर्थन प्राप्त गरिनेछ ।<br></code> | <code>[-0.06200315058231354, -0.016507938504219055, -0.029924314469099045, -0.052509162575006485, 0.07746178656816483, ...]</code> |
297
+ | <code>Polimatrix, Inc. is a system integrator and total solutions provider delivering radiation and nuclear protection and detection.<br></code> | <code>पोलिमाट्रिक्स, इन्कर्पोरेटिड प्रणाली इन्टिजर र कुल समाधान प्रदायक रेडियो र आणविक संरक्षण र पत्ता लगाउने प्रणाली इन्टिजर र कुल समाधान प्रदायक हो।<br></code> | <code>[-0.0446796678006649, 0.026428330689668655, -0.09837698936462402, -0.07765442878007889, -0.020364686846733093, ...]</code> |
298
+ * Loss: [<code>MSELoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#mseloss)
299
+
300
+ ### Training Hyperparameters
301
+ #### Non-Default Hyperparameters
302
+
303
+ - `eval_strategy`: steps
304
+ - `per_device_train_batch_size`: 64
305
+ - `per_device_eval_batch_size`: 64
306
+ - `learning_rate`: 2e-05
307
+ - `num_train_epochs`: 1
308
+ - `warmup_ratio`: 0.1
309
+ - `bf16`: True
310
+ - `push_to_hub`: True
311
+ - `hub_model_id`: jangedoo/all-MiniLM-L6-v2-nepali
312
+ - `push_to_hub_model_id`: all-MiniLM-L6-v2-nepali
313
+
314
+ #### All Hyperparameters
315
+ <details><summary>Click to expand</summary>
316
+
317
+ - `overwrite_output_dir`: False
318
+ - `do_predict`: False
319
+ - `eval_strategy`: steps
320
+ - `prediction_loss_only`: True
321
+ - `per_device_train_batch_size`: 64
322
+ - `per_device_eval_batch_size`: 64
323
+ - `per_gpu_train_batch_size`: None
324
+ - `per_gpu_eval_batch_size`: None
325
+ - `gradient_accumulation_steps`: 1
326
+ - `eval_accumulation_steps`: None
327
+ - `learning_rate`: 2e-05
328
+ - `weight_decay`: 0.0
329
+ - `adam_beta1`: 0.9
330
+ - `adam_beta2`: 0.999
331
+ - `adam_epsilon`: 1e-08
332
+ - `max_grad_norm`: 1.0
333
+ - `num_train_epochs`: 1
334
+ - `max_steps`: -1
335
+ - `lr_scheduler_type`: linear
336
+ - `lr_scheduler_kwargs`: {}
337
+ - `warmup_ratio`: 0.1
338
+ - `warmup_steps`: 0
339
+ - `log_level`: passive
340
+ - `log_level_replica`: warning
341
+ - `log_on_each_node`: True
342
+ - `logging_nan_inf_filter`: True
343
+ - `save_safetensors`: True
344
+ - `save_on_each_node`: False
345
+ - `save_only_model`: False
346
+ - `restore_callback_states_from_checkpoint`: False
347
+ - `no_cuda`: False
348
+ - `use_cpu`: False
349
+ - `use_mps_device`: False
350
+ - `seed`: 42
351
+ - `data_seed`: None
352
+ - `jit_mode_eval`: False
353
+ - `use_ipex`: False
354
+ - `bf16`: True
355
+ - `fp16`: False
356
+ - `fp16_opt_level`: O1
357
+ - `half_precision_backend`: auto
358
+ - `bf16_full_eval`: False
359
+ - `fp16_full_eval`: False
360
+ - `tf32`: None
361
+ - `local_rank`: 0
362
+ - `ddp_backend`: None
363
+ - `tpu_num_cores`: None
364
+ - `tpu_metrics_debug`: False
365
+ - `debug`: []
366
+ - `dataloader_drop_last`: False
367
+ - `dataloader_num_workers`: 0
368
+ - `dataloader_prefetch_factor`: None
369
+ - `past_index`: -1
370
+ - `disable_tqdm`: False
371
+ - `remove_unused_columns`: True
372
+ - `label_names`: None
373
+ - `load_best_model_at_end`: False
374
+ - `ignore_data_skip`: False
375
+ - `fsdp`: []
376
+ - `fsdp_min_num_params`: 0
377
+ - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
378
+ - `fsdp_transformer_layer_cls_to_wrap`: None
379
+ - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
380
+ - `deepspeed`: None
381
+ - `label_smoothing_factor`: 0.0
382
+ - `optim`: adamw_torch
383
+ - `optim_args`: None
384
+ - `adafactor`: False
385
+ - `group_by_length`: False
386
+ - `length_column_name`: length
387
+ - `ddp_find_unused_parameters`: None
388
+ - `ddp_bucket_cap_mb`: None
389
+ - `ddp_broadcast_buffers`: False
390
+ - `dataloader_pin_memory`: True
391
+ - `dataloader_persistent_workers`: False
392
+ - `skip_memory_metrics`: True
393
+ - `use_legacy_prediction_loop`: False
394
+ - `push_to_hub`: True
395
+ - `resume_from_checkpoint`: None
396
+ - `hub_model_id`: jangedoo/all-MiniLM-L6-v2-nepali
397
+ - `hub_strategy`: every_save
398
+ - `hub_private_repo`: False
399
+ - `hub_always_push`: False
400
+ - `gradient_checkpointing`: False
401
+ - `gradient_checkpointing_kwargs`: None
402
+ - `include_inputs_for_metrics`: False
403
+ - `eval_do_concat_batches`: True
404
+ - `fp16_backend`: auto
405
+ - `push_to_hub_model_id`: all-MiniLM-L6-v2-nepali
406
+ - `push_to_hub_organization`: None
407
+ - `mp_parameters`:
408
+ - `auto_find_batch_size`: False
409
+ - `full_determinism`: False
410
+ - `torchdynamo`: None
411
+ - `ray_scope`: last
412
+ - `ddp_timeout`: 1800
413
+ - `torch_compile`: False
414
+ - `torch_compile_backend`: None
415
+ - `torch_compile_mode`: None
416
+ - `dispatch_batches`: None
417
+ - `split_batches`: None
418
+ - `include_tokens_per_second`: False
419
+ - `include_num_input_tokens_seen`: False
420
+ - `neftune_noise_alpha`: None
421
+ - `optim_target_modules`: None
422
+ - `batch_eval_metrics`: False
423
+ - `eval_on_start`: False
424
+ - `batch_sampler`: batch_sampler
425
+ - `multi_dataset_batch_sampler`: proportional
426
+
427
+ </details>
428
+
429
+ ### Training Logs
430
+ | Epoch | Step | Training Loss | loss | mean_accuracy | negative_mse |
431
+ |:-----:|:----:|:-------------:|:------:|:-------------:|:------------:|
432
+ | 0.4 | 50 | 0.0021 | 0.0019 | 0.0111 | -0.3837 |
433
+ | 0.8 | 100 | 0.002 | 0.0019 | 0.0123 | -0.3794 |
434
+ | 0.4 | 50 | 0.002 | 0.0019 | 0.0130 | -0.3773 |
435
+ | 0.8 | 100 | 0.002 | 0.0019 | 0.0135 | -0.3744 |
436
+
437
+
438
+ ### Framework Versions
439
+ - Python: 3.10.12
440
+ - Sentence Transformers: 3.0.1
441
+ - Transformers: 4.42.4
442
+ - PyTorch: 2.3.1+cu121
443
+ - Accelerate: 0.32.1
444
+ - Datasets: 2.21.0
445
+ - Tokenizers: 0.19.1
446
+
447
+ ## Citation
448
+
449
+ ### BibTeX
450
+
451
+ #### Sentence Transformers
452
+ ```bibtex
453
+ @inproceedings{reimers-2019-sentence-bert,
454
+ title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
455
+ author = "Reimers, Nils and Gurevych, Iryna",
456
+ booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
457
+ month = "11",
458
+ year = "2019",
459
+ publisher = "Association for Computational Linguistics",
460
+ url = "https://arxiv.org/abs/1908.10084",
461
+ }
462
+ ```
463
+
464
+ #### MSELoss
465
+ ```bibtex
466
+ @inproceedings{reimers-2020-multilingual-sentence-bert,
467
+ title = "Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation",
468
+ author = "Reimers, Nils and Gurevych, Iryna",
469
+ booktitle = "Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing",
470
+ month = "11",
471
+ year = "2020",
472
+ publisher = "Association for Computational Linguistics",
473
+ url = "https://arxiv.org/abs/2004.09813",
474
+ }
475
+ ```
476
+
477
+ <!--
478
+ ## Glossary
479
+
480
+ *Clearly define terms in order to be accessible across audiences.*
481
+ -->
482
+
483
+ <!--
484
+ ## Model Card Authors
485
+
486
+ *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
487
+ -->
488
+
489
+ <!--
490
+ ## Model Card Contact
491
+
492
+ *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
493
+ -->
config_sentence_transformers.json ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "__version__": {
3
+ "sentence_transformers": "3.0.1",
4
+ "transformers": "4.42.4",
5
+ "pytorch": "2.3.1+cu121"
6
+ },
7
+ "prompts": {},
8
+ "default_prompt_name": null,
9
+ "similarity_fn_name": null
10
+ }
modules.json ADDED
@@ -0,0 +1,20 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [
2
+ {
3
+ "idx": 0,
4
+ "name": "0",
5
+ "path": "",
6
+ "type": "sentence_transformers.models.Transformer"
7
+ },
8
+ {
9
+ "idx": 1,
10
+ "name": "1",
11
+ "path": "1_Pooling",
12
+ "type": "sentence_transformers.models.Pooling"
13
+ },
14
+ {
15
+ "idx": 2,
16
+ "name": "2",
17
+ "path": "2_Normalize",
18
+ "type": "sentence_transformers.models.Normalize"
19
+ }
20
+ ]
sentence_bert_config.json ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ {
2
+ "max_seq_length": 256,
3
+ "do_lower_case": false
4
+ }