metadata

base_model: Alibaba-NLP/gte-base-en-v1.5
datasets: []
language:
  - en
library_name: sentence-transformers
license: apache-2.0
metrics:
  - cosine_accuracy@1
  - cosine_accuracy@3
  - cosine_accuracy@5
  - cosine_accuracy@10
  - cosine_precision@1
  - cosine_precision@3
  - cosine_precision@5
  - cosine_precision@10
  - cosine_recall@1
  - cosine_recall@3
  - cosine_recall@5
  - cosine_recall@10
  - cosine_ndcg@10
  - cosine_mrr@10
  - cosine_map@100
pipeline_tag: sentence-similarity
tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - generated_from_trainer
  - dataset_size:32833
  - loss:MatryoshkaLoss
  - loss:MultipleNegativesRankingLoss
widget:
  - source_sentence: >-
      Anonymity in online interactions can lead to a disinhibition effect, where
      individuals feel free to express hostile or aggressive opinions they might
      otherwise suppress.
    sentences:
      - What are the implications of anonymity in online interactions?
      - >-
        How does creativity function as a form of costly signalling in personal
        expressions such as invitations?
      - Why is conflict considered essential in a creative organization?
  - source_sentence: >-
      The author decides to release their novel into the world despite its
      imperfections, and finds that this allows them to move on to new projects
      and experiences, and to focus on the value of the work itself rather than
      its flaws.
    sentences:
      - >-
        How does the author's experience with their novel illustrate the concept
        of 'embracing imperfection' in creative work?
      - >-
        What does the author mean by 'ambitious programmers are better off doing
        their own thing'?
      - What is the role of 'show me' in the design process?
  - source_sentence: >-
      Tokens become more valuable as more users adopt them, creating a positive
      feedback loop that enhances their utility and encourages further adoption
      across various applications.
    sentences:
      - In what ways do tokens exhibit network effects?
      - >-
        What can sometimes be found when considering a startup with a
        lame-sounding idea?
      - >-
        How do social norms influence decision-making in the context of airport
        choices?
  - source_sentence: >-
      Philosophers are often viewed as the guardians of critical thinking;
      however, their reliance on bureaucratic structures and abstract
      discussions can become problematic. Instead of fostering open-mindedness,
      they may perpetuate dogmatic thinking and limit the exploration of diverse
      perspectives, thereby failing to fulfill their duty of promoting genuine
      critical engagement.
    sentences:
      - >-
        In what ways can the role of philosophers be seen as essential or
        problematic within the context of critical thinking?
      - >-
        How does the evolution of pair-bonding facilitate cultural exchange
        between groups?
      - What is the role of autonomy in the success of acquired startups?
  - source_sentence: >-
      Society tends to admire those who despair when others hope, viewing them
      as sages or wise figures.
    sentences:
      - >-
        What is often the societal perception of those who express pessimism
        about the future?
      - >-
        How did the realization about user engagement influence the app
        development strategy?
      - >-
        What lessons can be learned from the historical context of employee
        relations in large corporations?
model-index:
  - name: Custom Embedding Test - Anudit Nagar
    results:
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 768
          type: dim_768
        metrics:
          - type: cosine_accuracy@1
            value: 0.7683027145599123
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.8755141211955032
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.9097888675623801
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.9465313956676721
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.7683027145599123
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.29183804039850103
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.18195777351247602
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.09465313956676721
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.7683027145599123
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.8755141211955032
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.9097888675623801
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.9465313956676721
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.8566925927271383
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.8279207524340517
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.8302321946792381
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 512
          type: dim_512
        metrics:
          - type: cosine_accuracy@1
            value: 0.762818755141212
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.8700301617768028
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.9062242939402249
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.946257197696737
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.762818755141212
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.2900100539256009
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.18124485878804497
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.09462571976967371
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.762818755141212
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.8700301617768028
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.9062242939402249
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.946257197696737
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.8529743473843932
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.8231949721667308
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.825407004380477
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 256
          type: dim_256
        metrics:
          - type: cosine_accuracy@1
            value: 0.762818755141212
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.8683849739511927
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.9015629284343296
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.9418700301617768
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.762818755141212
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.28946165798373086
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.18031258568686592
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.09418700301617768
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.762818755141212
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.8683849739511927
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.9015629284343296
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.9418700301617768
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.850685453111757
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.8215859088357048
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.8239714751253995
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 128
          type: dim_128
        metrics:
          - type: cosine_accuracy@1
            value: 0.7573347957225116
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.8634494104743625
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.8952563751028242
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.9347408829174664
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.7573347957225116
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.2878164701581208
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.17905127502056484
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.09347408829174664
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.7573347957225116
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.8634494104743625
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.8952563751028242
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.9347408829174664
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.8445055968214926
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.8157123053956075
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.8184088689781863
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 64
          type: dim_64
        metrics:
          - type: cosine_accuracy@1
            value: 0.7419797093501508
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.8530298875788319
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.8859336440910337
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.9284343295859611
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.7419797093501508
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.28434329585961066
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.17718672881820677
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.09284343295859611
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.7419797093501508
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.8530298875788319
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.8859336440910337
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.9284343295859611
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.8334906130922063
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.8032139919307455
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.8057146368194794
            name: Cosine Map@100

Custom Embedding Test - Anudit Nagar

This is a sentence-transformers model finetuned from Alibaba-NLP/gte-base-en-v1.5. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: Alibaba-NLP/gte-base-en-v1.5
Maximum Sequence Length: 8192 tokens
Output Dimensionality: 768 tokens
Similarity Function: Cosine Similarity
Language: en
License: apache-2.0

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False}) with Transformer model: NewModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'Society tends to admire those who despair when others hope, viewing them as sages or wise figures.',
    'What is often the societal perception of those who express pessimism about the future?',
    'How did the realization about user engagement influence the app development strategy?',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Information Retrieval

Dataset: dim_768
Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.7683
cosine_accuracy@3	0.8755
cosine_accuracy@5	0.9098
cosine_accuracy@10	0.9465
cosine_precision@1	0.7683
cosine_precision@3	0.2918
cosine_precision@5	0.182
cosine_precision@10	0.0947
cosine_recall@1	0.7683
cosine_recall@3	0.8755
cosine_recall@5	0.9098
cosine_recall@10	0.9465
cosine_ndcg@10	0.8567
cosine_mrr@10	0.8279
cosine_map@100	0.8302

Information Retrieval

Dataset: dim_512
Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.7628
cosine_accuracy@3	0.87
cosine_accuracy@5	0.9062
cosine_accuracy@10	0.9463
cosine_precision@1	0.7628
cosine_precision@3	0.29
cosine_precision@5	0.1812
cosine_precision@10	0.0946
cosine_recall@1	0.7628
cosine_recall@3	0.87
cosine_recall@5	0.9062
cosine_recall@10	0.9463
cosine_ndcg@10	0.853
cosine_mrr@10	0.8232
cosine_map@100	0.8254

Information Retrieval

Dataset: dim_256
Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.7628
cosine_accuracy@3	0.8684
cosine_accuracy@5	0.9016
cosine_accuracy@10	0.9419
cosine_precision@1	0.7628
cosine_precision@3	0.2895
cosine_precision@5	0.1803
cosine_precision@10	0.0942
cosine_recall@1	0.7628
cosine_recall@3	0.8684
cosine_recall@5	0.9016
cosine_recall@10	0.9419
cosine_ndcg@10	0.8507
cosine_mrr@10	0.8216
cosine_map@100	0.824

Information Retrieval

Dataset: dim_128
Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.7573
cosine_accuracy@3	0.8634
cosine_accuracy@5	0.8953
cosine_accuracy@10	0.9347
cosine_precision@1	0.7573
cosine_precision@3	0.2878
cosine_precision@5	0.1791
cosine_precision@10	0.0935
cosine_recall@1	0.7573
cosine_recall@3	0.8634
cosine_recall@5	0.8953
cosine_recall@10	0.9347
cosine_ndcg@10	0.8445
cosine_mrr@10	0.8157
cosine_map@100	0.8184

Information Retrieval

Dataset: dim_64
Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.742
cosine_accuracy@3	0.853
cosine_accuracy@5	0.8859
cosine_accuracy@10	0.9284
cosine_precision@1	0.742
cosine_precision@3	0.2843
cosine_precision@5	0.1772
cosine_precision@10	0.0928
cosine_recall@1	0.742
cosine_recall@3	0.853
cosine_recall@5	0.8859
cosine_recall@10	0.9284
cosine_ndcg@10	0.8335
cosine_mrr@10	0.8032
cosine_map@100	0.8057

Training Details

Training Dataset

Unnamed Dataset

Size: 32,833 training samples
Columns: positive and anchor
Approximate statistics based on the first 1000 samples:
positive anchor
type string string
details
min: 3 tokens
mean: 34.54 tokens
max: 102 tokens

min: 9 tokens
mean: 16.78 tokens
max: 77 tokens

	positive	anchor
type	string	string
details	min: 3 tokens mean: 34.54 tokens max: 102 tokens	min: 9 tokens mean: 16.78 tokens max: 77 tokens

Samples:

positive	anchor
`The author saw taking risks as a necessary part of the creative process, and was willing to take risks in order to explore new ideas and themes.`	`What was the author's perspective on the importance of taking risks in creative work?`
`Recognizing that older users are less likely to invite new users led to a strategic focus on younger demographics, prompting a shift in development efforts toward creating products that resonate with teens.`	`How did the realization about user engagement influence the app development strategy?`
`The phrase emphasizes the fragility of Earth and our collective responsibility to protect it and ensure sustainable resource management for future generations.`	`What is the significance of the phrase 'pale blue dot' in relation to environmental responsibility?`

Loss: MatryoshkaLoss with these parameters:

{
    "loss": "MultipleNegativesRankingLoss",
    "matryoshka_dims": [
        768,
        512,
        256,
        128,
        64
    ],
    "matryoshka_weights": [
        1,
        1,
        1,
        1,
        1
    ],
    "n_dims_per_step": -1
}

Training Hyperparameters

Non-Default Hyperparameters

eval_strategy: epoch
per_device_train_batch_size: 32
per_device_eval_batch_size: 16
gradient_accumulation_steps: 16
learning_rate: 0.0002
num_train_epochs: 5
lr_scheduler_type: cosine
warmup_ratio: 0.1
bf16: True
load_best_model_at_end: True
batch_sampler: no_duplicates

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: epoch
prediction_loss_only: True
per_device_train_batch_size: 32
per_device_eval_batch_size: 16
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 16
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 0.0002
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1.0
num_train_epochs: 5
max_steps: -1
lr_scheduler_type: cosine
lr_scheduler_kwargs: {}
warmup_ratio: 0.1
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
use_ipex: False
bf16: True
fp16: False
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: True
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: False
hub_always_push: False
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
dispatch_batches: None
split_batches: None
include_tokens_per_second: False
include_num_input_tokens_seen: False
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
eval_use_gather_object: False
batch_sampler: no_duplicates
multi_dataset_batch_sampler: proportional

Training Logs

Epoch	Step	Training Loss	dim_128_cosine_map@100	dim_256_cosine_map@100	dim_512_cosine_map@100	dim_64_cosine_map@100	dim_768_cosine_map@100
0.1558	10	0.7195	-	-	-	-	-
0.3116	20	0.324	-	-	-	-	-
0.4674	30	0.238	-	-	-	-	-
0.6232	40	0.2265	-	-	-	-	-
0.7790	50	0.1825	-	-	-	-	-
0.9348	60	0.1938	-	-	-	-	-
0.9971	64	-	0.8054	0.8198	0.8276	0.7796	0.8329
1.0906	70	0.1397	-	-	-	-	-
1.2463	80	0.0611	-	-	-	-	-
1.4021	90	0.0506	-	-	-	-	-
1.5579	100	0.047	-	-	-	-	-
1.7137	110	0.0327	-	-	-	-	-
1.8695	120	0.034	-	-	-	-	-
1.9942	128	-	0.8036	0.8135	0.8187	0.7861	0.8243
2.0253	130	0.0319	-	-	-	-	-
2.1811	140	0.0347	-	-	-	-	-
2.3369	150	0.021	-	-	-	-	-
2.4927	160	0.0169	-	-	-	-	-
2.6485	170	0.0135	-	-	-	-	-
2.8043	180	0.0123	-	-	-	-	-
2.9601	190	0.0111	-	-	-	-	-
2.9912	192	-	0.8109	0.8179	0.8213	0.7973	0.8264
3.1159	200	0.0083	-	-	-	-	-
3.2717	210	0.0088	-	-	-	-	-
3.4275	220	0.005	-	-	-	-	-
3.5833	230	0.005	-	-	-	-	-
3.7390	240	0.0043	-	-	-	-	-
3.8948	250	0.0058	-	-	-	-	-
3.9883	256	-	0.8163	0.8244	0.8260	0.8045	0.8287
4.0506	260	0.0057	-	-	-	-	-
4.2064	270	0.0035	-	-	-	-	-
4.3622	280	0.0033	-	-	-	-	-
4.5180	290	0.0032	-	-	-	-	-
4.6738	300	0.0031	-	-	-	-	-
4.8296	310	0.0038	-	-	-	-	-
4.9854	320	0.0042	0.8184	0.8240	0.8254	0.8057	0.8302

The bold row denotes the saved checkpoint.

Framework Versions

Python: 3.12.5
Sentence Transformers: 3.0.1
Transformers: 4.44.2
PyTorch: 2.4.0
Accelerate: 0.33.0
Datasets: 2.21.0
Tokenizers: 0.19.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning}, 
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply}, 
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}