Hugging Face's logo

NickyNicky
/
StaticEmbedding-MatryoshkaLoss-gemma-2-2b-en-es

Sentence Similarity
sentence-transformers
Safetensors
feature-extraction
Generated from Trainer
dataset_size:4322286
loss:MatryoshkaLoss
loss:MultipleNegativesRankingLoss
Inference Endpoints
Model card Files Community
NickyNicky commited on
Commit
24d8041
·
verified ·
1 Parent(s): 62c5c82

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +240 -0
README.md CHANGED
@@ -434,6 +434,246 @@ You can finetune this model on your own dataset.
434
  - Datasets: 3.2.0
435
  - Tokenizers: 0.21.0
436
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
437
  ## Citation
438
 
439
  ### BibTeX
 
434
  - Datasets: 3.2.0
435
  - Tokenizers: 0.21.0
436
 
437
+
438
+ ## NanoBEIREvaluator > 0.8
439
+ ```
440
+ {
441
+ "NanoDBPedia_cosine_accuracy@3": 0.86,
442
+ "NanoDBPedia_cosine_accuracy@5": 0.92,
443
+ "NanoDBPedia_cosine_accuracy@10": 0.96,
444
+ "NanoFEVER_cosine_accuracy@3": 0.86,
445
+ "NanoFEVER_cosine_accuracy@5": 0.92,
446
+ "NanoFEVER_cosine_accuracy@10": 0.96,
447
+ "NanoQuoraRetrieval_cosine_accuracy@1": 0.88,
448
+ "NanoQuoraRetrieval_cosine_accuracy@3": 0.96,
449
+ "NanoQuoraRetrieval_cosine_accuracy@5": 1.0,
450
+ "NanoQuoraRetrieval_cosine_accuracy@10": 1.0,
451
+ "NanoSCIDOCS_cosine_accuracy@5": 0.82,
452
+ "NanoSCIDOCS_cosine_accuracy@10": 0.92,
453
+ "NanoArguAna_cosine_accuracy@10": 0.92,
454
+ "NanoSciFact_cosine_accuracy@10": 0.88,
455
+ "NanoHotpotQA_cosine_accuracy@10": 0.88,
456
+ "NanoTouche2020_cosine_accuracy@5": 0.9183673469387755,
457
+ "NanoTouche2020_cosine_accuracy@10": 0.9387755102040817,
458
+ "NanoBEIR_mean_cosine_accuracy@10": 0.8583673469387756
459
+ }
460
+ ```
461
+
462
+
463
+ ## All NanoBEIREvaluator
464
+ ```
465
+ {'NanoClimateFEVER_cosine_accuracy@1': 0.28,
466
+ 'NanoClimateFEVER_cosine_accuracy@3': 0.44,
467
+ 'NanoClimateFEVER_cosine_accuracy@5': 0.54,
468
+ 'NanoClimateFEVER_cosine_accuracy@10': 0.72,
469
+ 'NanoClimateFEVER_cosine_precision@1': 0.28,
470
+ 'NanoClimateFEVER_cosine_precision@3': 0.15333333333333332,
471
+ 'NanoClimateFEVER_cosine_precision@5': 0.124,
472
+ 'NanoClimateFEVER_cosine_precision@10': 0.08999999999999998,
473
+ 'NanoClimateFEVER_cosine_recall@1': 0.145,
474
+ 'NanoClimateFEVER_cosine_recall@3': 0.205,
475
+ 'NanoClimateFEVER_cosine_recall@5': 0.264,
476
+ 'NanoClimateFEVER_cosine_recall@10': 0.36200000000000004,
477
+ 'NanoClimateFEVER_cosine_ndcg@10': 0.2957527689242254,
478
+ 'NanoClimateFEVER_cosine_mrr@10': 0.3996666666666668,
479
+ 'NanoClimateFEVER_cosine_map@100': 0.23258384801937396,
480
+ 'NanoDBPedia_cosine_accuracy@1': 0.68,
481
+ 'NanoDBPedia_cosine_accuracy@3': 0.86,
482
+ 'NanoDBPedia_cosine_accuracy@5': 0.92,
483
+ 'NanoDBPedia_cosine_accuracy@10': 0.96,
484
+ 'NanoDBPedia_cosine_precision@1': 0.68,
485
+ 'NanoDBPedia_cosine_precision@3': 0.56,
486
+ 'NanoDBPedia_cosine_precision@5': 0.5120000000000001,
487
+ 'NanoDBPedia_cosine_precision@10': 0.43800000000000006,
488
+ 'NanoDBPedia_cosine_recall@1': 0.07601531530835434,
489
+ 'NanoDBPedia_cosine_recall@3': 0.1438904710839341,
490
+ 'NanoDBPedia_cosine_recall@5': 0.20681359525684506,
491
+ 'NanoDBPedia_cosine_recall@10': 0.319966975132044,
492
+ 'NanoDBPedia_cosine_ndcg@10': 0.5501100350453579,
493
+ 'NanoDBPedia_cosine_mrr@10': 0.7855000000000001,
494
+ 'NanoDBPedia_cosine_map@100': 0.39476156890024533,
495
+ 'NanoFEVER_cosine_accuracy@1': 0.68,
496
+ 'NanoFEVER_cosine_accuracy@3': 0.86,
497
+ 'NanoFEVER_cosine_accuracy@5': 0.92,
498
+ 'NanoFEVER_cosine_accuracy@10': 0.96,
499
+ 'NanoFEVER_cosine_precision@1': 0.68,
500
+ 'NanoFEVER_cosine_precision@3': 0.29333333333333333,
501
+ 'NanoFEVER_cosine_precision@5': 0.19199999999999995,
502
+ 'NanoFEVER_cosine_precision@10': 0.10199999999999998,
503
+ 'NanoFEVER_cosine_recall@1': 0.6266666666666666,
504
+ 'NanoFEVER_cosine_recall@3': 0.8133333333333332,
505
+ 'NanoFEVER_cosine_recall@5': 0.8833333333333333,
506
+ 'NanoFEVER_cosine_recall@10': 0.9233333333333333,
507
+ 'NanoFEVER_cosine_ndcg@10': 0.7933479848498471,
508
+ 'NanoFEVER_cosine_mrr@10': 0.7780793650793651,
509
+ 'NanoFEVER_cosine_map@100': 0.7406571665049926,
510
+ 'NanoFiQA2018_cosine_accuracy@1': 0.46,
511
+ 'NanoFiQA2018_cosine_accuracy@3': 0.64,
512
+ 'NanoFiQA2018_cosine_accuracy@5': 0.7,
513
+ 'NanoFiQA2018_cosine_accuracy@10': 0.72,
514
+ 'NanoFiQA2018_cosine_precision@1': 0.46,
515
+ 'NanoFiQA2018_cosine_precision@3': 0.2866666666666666,
516
+ 'NanoFiQA2018_cosine_precision@5': 0.22399999999999998,
517
+ 'NanoFiQA2018_cosine_precision@10': 0.12999999999999998,
518
+ 'NanoFiQA2018_cosine_recall@1': 0.23924603174603173,
519
+ 'NanoFiQA2018_cosine_recall@3': 0.4251031746031746,
520
+ 'NanoFiQA2018_cosine_recall@5': 0.5099603174603174,
521
+ 'NanoFiQA2018_cosine_recall@10': 0.566015873015873,
522
+ 'NanoFiQA2018_cosine_ndcg@10': 0.4774545077577204,
523
+ 'NanoFiQA2018_cosine_mrr@10': 0.5475555555555556,
524
+ 'NanoFiQA2018_cosine_map@100': 0.4125452702654584,
525
+ 'NanoHotpotQA_cosine_accuracy@1': 0.64,
526
+ 'NanoHotpotQA_cosine_accuracy@3': 0.82,
527
+ 'NanoHotpotQA_cosine_accuracy@5': 0.84,
528
+ 'NanoHotpotQA_cosine_accuracy@10': 0.88,
529
+ 'NanoHotpotQA_cosine_precision@1': 0.64,
530
+ 'NanoHotpotQA_cosine_precision@3': 0.3533333333333333,
531
+ 'NanoHotpotQA_cosine_precision@5': 0.23599999999999993,
532
+ 'NanoHotpotQA_cosine_precision@10': 0.128,
533
+ 'NanoHotpotQA_cosine_recall@1': 0.32,
534
+ 'NanoHotpotQA_cosine_recall@3': 0.53,
535
+ 'NanoHotpotQA_cosine_recall@5': 0.59,
536
+ 'NanoHotpotQA_cosine_recall@10': 0.64,
537
+ 'NanoHotpotQA_cosine_ndcg@10': 0.5959681682828366,
538
+ 'NanoHotpotQA_cosine_mrr@10': 0.723888888888889,
539
+ 'NanoHotpotQA_cosine_map@100': 0.5262469568756968,
540
+ 'NanoMSMARCO_cosine_accuracy@1': 0.36,
541
+ 'NanoMSMARCO_cosine_accuracy@3': 0.52,
542
+ 'NanoMSMARCO_cosine_accuracy@5': 0.58,
543
+ 'NanoMSMARCO_cosine_accuracy@10': 0.8,
544
+ 'NanoMSMARCO_cosine_precision@1': 0.36,
545
+ 'NanoMSMARCO_cosine_precision@3': 0.1733333333333333,
546
+ 'NanoMSMARCO_cosine_precision@5': 0.11599999999999999,
547
+ 'NanoMSMARCO_cosine_precision@10': 0.08,
548
+ 'NanoMSMARCO_cosine_recall@1': 0.36,
549
+ 'NanoMSMARCO_cosine_recall@3': 0.52,
550
+ 'NanoMSMARCO_cosine_recall@5': 0.58,
551
+ 'NanoMSMARCO_cosine_recall@10': 0.8,
552
+ 'NanoMSMARCO_cosine_ndcg@10': 0.5539831330912274,
553
+ 'NanoMSMARCO_cosine_mrr@10': 0.47960317460317464,
554
+ 'NanoMSMARCO_cosine_map@100': 0.4907628900864195,
555
+ 'NanoNFCorpus_cosine_accuracy@1': 0.42,
556
+ 'NanoNFCorpus_cosine_accuracy@3': 0.56,
557
+ 'NanoNFCorpus_cosine_accuracy@5': 0.6,
558
+ 'NanoNFCorpus_cosine_accuracy@10': 0.7,
559
+ 'NanoNFCorpus_cosine_precision@1': 0.42,
560
+ 'NanoNFCorpus_cosine_precision@3': 0.3466666666666666,
561
+ 'NanoNFCorpus_cosine_precision@5': 0.32800000000000007,
562
+ 'NanoNFCorpus_cosine_precision@10': 0.286,
563
+ 'NanoNFCorpus_cosine_recall@1': 0.03391318439564492,
564
+ 'NanoNFCorpus_cosine_recall@3': 0.06311668492872162,
565
+ 'NanoNFCorpus_cosine_recall@5': 0.08191277059586696,
566
+ 'NanoNFCorpus_cosine_recall@10': 0.13476845853527392,
567
+ 'NanoNFCorpus_cosine_ndcg@10': 0.3322933792371396,
568
+ 'NanoNFCorpus_cosine_mrr@10': 0.4983333333333333,
569
+ 'NanoNFCorpus_cosine_map@100': 0.13985354018581944,
570
+ 'NanoNQ_cosine_accuracy@1': 0.44,
571
+ 'NanoNQ_cosine_accuracy@3': 0.64,
572
+ 'NanoNQ_cosine_accuracy@5': 0.66,
573
+ 'NanoNQ_cosine_accuracy@10': 0.76,
574
+ 'NanoNQ_cosine_precision@1': 0.44,
575
+ 'NanoNQ_cosine_precision@3': 0.22,
576
+ 'NanoNQ_cosine_precision@5': 0.14,
577
+ 'NanoNQ_cosine_precision@10': 0.08199999999999999,
578
+ 'NanoNQ_cosine_recall@1': 0.42,
579
+ 'NanoNQ_cosine_recall@3': 0.62,
580
+ 'NanoNQ_cosine_recall@5': 0.64,
581
+ 'NanoNQ_cosine_recall@10': 0.75,
582
+ 'NanoNQ_cosine_ndcg@10': 0.5903874296113161,
583
+ 'NanoNQ_cosine_mrr@10': 0.5456349206349206,
584
+ 'NanoNQ_cosine_map@100': 0.5437440035864959,
585
+ 'NanoQuoraRetrieval_cosine_accuracy@1': 0.88,
586
+ 'NanoQuoraRetrieval_cosine_accuracy@3': 0.96,
587
+ 'NanoQuoraRetrieval_cosine_accuracy@5': 1.0,
588
+ 'NanoQuoraRetrieval_cosine_accuracy@10': 1.0,
589
+ 'NanoQuoraRetrieval_cosine_precision@1': 0.88,
590
+ 'NanoQuoraRetrieval_cosine_precision@3': 0.3933333333333333,
591
+ 'NanoQuoraRetrieval_cosine_precision@5': 0.256,
592
+ 'NanoQuoraRetrieval_cosine_precision@10': 0.13599999999999998,
593
+ 'NanoQuoraRetrieval_cosine_recall@1': 0.784,
594
+ 'NanoQuoraRetrieval_cosine_recall@3': 0.9186666666666667,
595
+ 'NanoQuoraRetrieval_cosine_recall@5': 0.976,
596
+ 'NanoQuoraRetrieval_cosine_recall@10': 0.9933333333333334,
597
+ 'NanoQuoraRetrieval_cosine_ndcg@10': 0.9367841595958026,
598
+ 'NanoQuoraRetrieval_cosine_mrr@10': 0.9246666666666666,
599
+ 'NanoQuoraRetrieval_cosine_map@100': 0.913554834054834,
600
+ 'NanoSCIDOCS_cosine_accuracy@1': 0.52,
601
+ 'NanoSCIDOCS_cosine_accuracy@3': 0.68,
602
+ 'NanoSCIDOCS_cosine_accuracy@5': 0.82,
603
+ 'NanoSCIDOCS_cosine_accuracy@10': 0.92,
604
+ 'NanoSCIDOCS_cosine_precision@1': 0.52,
605
+ 'NanoSCIDOCS_cosine_precision@3': 0.3933333333333333,
606
+ 'NanoSCIDOCS_cosine_precision@5': 0.33599999999999997,
607
+ 'NanoSCIDOCS_cosine_precision@10': 0.21600000000000003,
608
+ 'NanoSCIDOCS_cosine_recall@1': 0.10966666666666666,
609
+ 'NanoSCIDOCS_cosine_recall@3': 0.24466666666666664,
610
+ 'NanoSCIDOCS_cosine_recall@5': 0.34566666666666657,
611
+ 'NanoSCIDOCS_cosine_recall@10': 0.44266666666666665,
612
+ 'NanoSCIDOCS_cosine_ndcg@10': 0.4328110226758414,
613
+ 'NanoSCIDOCS_cosine_mrr@10': 0.6317222222222222,
614
+ 'NanoSCIDOCS_cosine_map@100': 0.34997841607847063,
615
+ 'NanoArguAna_cosine_accuracy@1': 0.2,
616
+ 'NanoArguAna_cosine_accuracy@3': 0.56,
617
+ 'NanoArguAna_cosine_accuracy@5': 0.76,
618
+ 'NanoArguAna_cosine_accuracy@10': 0.92,
619
+ 'NanoArguAna_cosine_precision@1': 0.2,
620
+ 'NanoArguAna_cosine_precision@3': 0.18666666666666668,
621
+ 'NanoArguAna_cosine_precision@5': 0.15200000000000002,
622
+ 'NanoArguAna_cosine_precision@10': 0.092,
623
+ 'NanoArguAna_cosine_recall@1': 0.2,
624
+ 'NanoArguAna_cosine_recall@3': 0.56,
625
+ 'NanoArguAna_cosine_recall@5': 0.76,
626
+ 'NanoArguAna_cosine_recall@10': 0.92,
627
+ 'NanoArguAna_cosine_ndcg@10': 0.5499071039525992,
628
+ 'NanoArguAna_cosine_mrr@10': 0.43229365079365073,
629
+ 'NanoArguAna_cosine_map@100': 0.43523820792684886,
630
+ 'NanoSciFact_cosine_accuracy@1': 0.6,
631
+ 'NanoSciFact_cosine_accuracy@3': 0.72,
632
+ 'NanoSciFact_cosine_accuracy@5': 0.8,
633
+ 'NanoSciFact_cosine_accuracy@10': 0.88,
634
+ 'NanoSciFact_cosine_precision@1': 0.6,
635
+ 'NanoSciFact_cosine_precision@3': 0.25333333333333335,
636
+ 'NanoSciFact_cosine_precision@5': 0.18,
637
+ 'NanoSciFact_cosine_precision@10': 0.09799999999999999,
638
+ 'NanoSciFact_cosine_recall@1': 0.58,
639
+ 'NanoSciFact_cosine_recall@3': 0.7,
640
+ 'NanoSciFact_cosine_recall@5': 0.8,
641
+ 'NanoSciFact_cosine_recall@10': 0.87,
642
+ 'NanoSciFact_cosine_ndcg@10': 0.7265348054031264,
643
+ 'NanoSciFact_cosine_mrr@10': 0.6841031746031746,
644
+ 'NanoSciFact_cosine_map@100': 0.6810233866101422,
645
+ 'NanoTouche2020_cosine_accuracy@1': 0.5102040816326531,
646
+ 'NanoTouche2020_cosine_accuracy@3': 0.8367346938775511,
647
+ 'NanoTouche2020_cosine_accuracy@5': 0.9183673469387755,
648
+ 'NanoTouche2020_cosine_accuracy@10': 0.9387755102040817,
649
+ 'NanoTouche2020_cosine_precision@1': 0.5102040816326531,
650
+ 'NanoTouche2020_cosine_precision@3': 0.5374149659863945,
651
+ 'NanoTouche2020_cosine_precision@5': 0.5061224489795918,
652
+ 'NanoTouche2020_cosine_precision@10': 0.43265306122448977,
653
+ 'NanoTouche2020_cosine_recall@1': 0.03546508562664911,
654
+ 'NanoTouche2020_cosine_recall@3': 0.11189238805791148,
655
+ 'NanoTouche2020_cosine_recall@5': 0.1673503566176574,
656
+ 'NanoTouche2020_cosine_recall@10': 0.2818808841266296,
657
+ 'NanoTouche2020_cosine_ndcg@10': 0.47479704449085264,
658
+ 'NanoTouche2020_cosine_mrr@10': 0.6714285714285714,
659
+ 'NanoTouche2020_cosine_map@100': 0.3438320372291555,
660
+ 'NanoBEIR_mean_cosine_accuracy@1': 0.5130926216640502,
661
+ 'NanoBEIR_mean_cosine_accuracy@3': 0.6997488226059654,
662
+ 'NanoBEIR_mean_cosine_accuracy@5': 0.7737205651491367,
663
+ 'NanoBEIR_mean_cosine_accuracy@10': 0.8583673469387756,
664
+ 'NanoBEIR_mean_cosine_precision@1': 0.5130926216640502,
665
+ 'NanoBEIR_mean_cosine_precision@3': 0.31928833071690216,
666
+ 'NanoBEIR_mean_cosine_precision@5': 0.2540094191522763,
667
+ 'NanoBEIR_mean_cosine_precision@10': 0.1777425431711146,
668
+ 'NanoBEIR_mean_cosine_recall@1': 0.302305611570001,
669
+ 'NanoBEIR_mean_cosine_recall@3': 0.4504361065646467,
670
+ 'NanoBEIR_mean_cosine_recall@5': 0.5234643876869758,
671
+ 'NanoBEIR_mean_cosine_recall@10': 0.6156896557033196,
672
+ 'NanoBEIR_mean_cosine_ndcg@10': 0.5623178109936842,
673
+ 'NanoBEIR_mean_cosine_mrr@10': 0.6232673992673993,
674
+ 'NanoBEIR_mean_cosine_map@100': 0.47729093279415025}
675
+ ```
676
+
677
  ## Citation
678
 
679
  ### BibTeX