---
language:
- en
license: apache-2.0
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:6300
- loss:MatryoshkaLoss
- loss:MultipleNegativesRankingLoss
base_model: BAAI/bge-base-en-v1.5
widget:
- source_sentence: Consolidated Regulatory Capital - The capital requirements calculated
under the FRB’s Capital Framework include the capital conservation buffer requirements,
which are comprised of a 2.5% buffer (under the Advanced Capital Rules).
sentences:
- What was the effective income tax rate for the year ended December 31, 2023?
- What is the function of capital conservation buffer requirements in the FRB's
Capital Framework for banks like Group Inc. in 2023?
- What incentive does the Hawaiian Electric’s Battery Bonus grid services program
offer?
- source_sentence: Balance at beginning of year 2021 was $30 million and, after charge-offs,
recoveries, and provisions for credit losses, the balance at end of year was $18
million.
sentences:
- Between what dates did CS&Co allegedly violate their duty to seek best execution
as per the plaintiffs' allegations in the lawsuit involving UBS Securities LLC?
- What were the balance at the beginning and the end of the year for credit loss
balances in 2021?
- How does the company handle leasehold improvements in terms of depreciation?
- source_sentence: The Compute reporting unit has an excess of fair value over carrying
value of 5% as of the annual test date.
sentences:
- What percent excess of fair value over carrying value did the Compute reporting
unit have as of the annual test date in 2023?
- What were the effective income tax rates for fiscal years 2023, 2022, and 2021,
and how did specific tax events affect these rates?
- When does the latest expiring European composition of matter patent (Supplementary
Protection Certificate) for STELARA expire?
- source_sentence: The net revenue decrease during 2023 in the Entertainment segment
was driven by lower entertainment productions and deliveries, reflecting the impact
of the several months-long strikes during 2023 by the Writers Guild of America
and the American actors' union, SAG-AFTRA.
sentences:
- What was the operating income for Google Cloud in 2023?
- How much did the company contribute to its pension and OPEB plans in 2023?
- What was the impact of the strikes by the Writers Guild of America and SAG-AFTRA
on the Entertainment segment's net revenues in 2023?
- source_sentence: As a REIT, future repatriation of incremental undistributed earnings
of the company's foreign subsidiaries will not be subject to federal or state
income tax, with the exception of foreign withholding taxes.
sentences:
- What tax implications apply to the future repatriation of incremental undistributed
earnings by a REIT from its foreign subsidiaries?
- What was the accrued liability for product recall related matters as of the end
of the fiscal year on June 30, 2023?
- What was the total amount of future interest payments associated with the Notes
as of September 30, 2023?
pipeline_tag: sentence-similarity
library_name: sentence-transformers
metrics:
- cosine_accuracy@1
- cosine_accuracy@3
- cosine_accuracy@5
- cosine_accuracy@10
- cosine_precision@1
- cosine_precision@3
- cosine_precision@5
- cosine_precision@10
- cosine_recall@1
- cosine_recall@3
- cosine_recall@5
- cosine_recall@10
- cosine_ndcg@10
- cosine_mrr@10
- cosine_map@100
model-index:
- name: BGE base Financial Matryoshka
results:
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: dim 768
type: dim_768
metrics:
- type: cosine_accuracy@1
value: 0.7128571428571429
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.8428571428571429
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.88
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.92
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.7128571428571429
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.28095238095238095
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.176
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.09199999999999998
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.7128571428571429
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.8428571428571429
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.88
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.92
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.8194470096208256
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.7869285714285713
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.7892168694112985
name: Cosine Map@100
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: dim 512
type: dim_512
metrics:
- type: cosine_accuracy@1
value: 0.7214285714285714
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.8471428571428572
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.8757142857142857
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.9185714285714286
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.7214285714285714
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.2823809523809524
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.17514285714285713
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.09185714285714286
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.7214285714285714
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.8471428571428572
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.8757142857142857
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.9185714285714286
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.8222551376922121
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.7912256235827663
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.7935743687249276
name: Cosine Map@100
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: dim 256
type: dim_256
metrics:
- type: cosine_accuracy@1
value: 0.7042857142857143
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.8342857142857143
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.8771428571428571
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.9157142857142857
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.7042857142857143
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.27809523809523806
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.1754285714285714
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.09157142857142857
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.7042857142857143
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.8342857142857143
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.8771428571428571
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.9157142857142857
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.813165438848782
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.7800498866213152
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.7822651539071127
name: Cosine Map@100
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: dim 128
type: dim_128
metrics:
- type: cosine_accuracy@1
value: 0.6971428571428572
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.8142857142857143
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.8557142857142858
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.9028571428571428
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.6971428571428572
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.2714285714285714
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.17114285714285712
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.09028571428571427
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.6971428571428572
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.8142857142857143
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.8557142857142858
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.9028571428571428
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.7996582219917312
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.7667329931972787
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.7700915959452638
name: Cosine Map@100
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: dim 64
type: dim_64
metrics:
- type: cosine_accuracy@1
value: 0.6742857142857143
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.7942857142857143
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.8257142857142857
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.8742857142857143
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.6742857142857143
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.26476190476190475
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.16514285714285712
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.08742857142857141
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.6742857142857143
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.7942857142857143
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.8257142857142857
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.8742857142857143
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.7742733360934079
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.7424053287981859
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.7463231326238146
name: Cosine Map@100
---
# BGE base Financial Matryoshka
This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [BAAI/bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5) on the json dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
## Model Details
### Model Description
- **Model Type:** Sentence Transformer
- **Base model:** [BAAI/bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5)
- **Maximum Sequence Length:** 512 tokens
- **Output Dimensionality:** 768 dimensions
- **Similarity Function:** Cosine Similarity
- **Training Dataset:**
- json
- **Language:** en
- **License:** apache-2.0
### Model Sources
- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
### Full Model Architecture
```
SentenceTransformer(
(0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel
(1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
(2): Normalize()
)
```
## Usage
### Direct Usage (Sentence Transformers)
First install the Sentence Transformers library:
```bash
pip install -U sentence-transformers
```
Then you can load this model and run inference.
```python
from sentence_transformers import SentenceTransformer
# Download from the 🤗 Hub
model = SentenceTransformer("mezeidragos-lateral/bge-base-financial-matryoshka")
# Run inference
sentences = [
"As a REIT, future repatriation of incremental undistributed earnings of the company's foreign subsidiaries will not be subject to federal or state income tax, with the exception of foreign withholding taxes.",
'What tax implications apply to the future repatriation of incremental undistributed earnings by a REIT from its foreign subsidiaries?',
'What was the accrued liability for product recall related matters as of the end of the fiscal year on June 30, 2023?',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]
# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
```
## Evaluation
### Metrics
#### Information Retrieval
* Datasets: `dim_768`, `dim_512`, `dim_256`, `dim_128` and `dim_64`
* Evaluated with [InformationRetrievalEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)
| Metric | dim_768 | dim_512 | dim_256 | dim_128 | dim_64 |
|:--------------------|:-----------|:-----------|:-----------|:-----------|:-----------|
| cosine_accuracy@1 | 0.7129 | 0.7214 | 0.7043 | 0.6971 | 0.6743 |
| cosine_accuracy@3 | 0.8429 | 0.8471 | 0.8343 | 0.8143 | 0.7943 |
| cosine_accuracy@5 | 0.88 | 0.8757 | 0.8771 | 0.8557 | 0.8257 |
| cosine_accuracy@10 | 0.92 | 0.9186 | 0.9157 | 0.9029 | 0.8743 |
| cosine_precision@1 | 0.7129 | 0.7214 | 0.7043 | 0.6971 | 0.6743 |
| cosine_precision@3 | 0.281 | 0.2824 | 0.2781 | 0.2714 | 0.2648 |
| cosine_precision@5 | 0.176 | 0.1751 | 0.1754 | 0.1711 | 0.1651 |
| cosine_precision@10 | 0.092 | 0.0919 | 0.0916 | 0.0903 | 0.0874 |
| cosine_recall@1 | 0.7129 | 0.7214 | 0.7043 | 0.6971 | 0.6743 |
| cosine_recall@3 | 0.8429 | 0.8471 | 0.8343 | 0.8143 | 0.7943 |
| cosine_recall@5 | 0.88 | 0.8757 | 0.8771 | 0.8557 | 0.8257 |
| cosine_recall@10 | 0.92 | 0.9186 | 0.9157 | 0.9029 | 0.8743 |
| **cosine_ndcg@10** | **0.8194** | **0.8223** | **0.8132** | **0.7997** | **0.7743** |
| cosine_mrr@10 | 0.7869 | 0.7912 | 0.78 | 0.7667 | 0.7424 |
| cosine_map@100 | 0.7892 | 0.7936 | 0.7823 | 0.7701 | 0.7463 |
## Training Details
### Training Dataset
#### json
* Dataset: json
* Size: 6,300 training samples
* Columns: positive and anchor
* Approximate statistics based on the first 1000 samples:
| | positive | anchor |
|:--------|:-----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|
| type | string | string |
| details | - min: 4 tokens
- mean: 44.62 tokens
- max: 301 tokens
| - min: 10 tokens
- mean: 20.66 tokens
- max: 45 tokens
|
* Samples:
| positive | anchor |
|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------|
| We provide transaction processing services (primarily authorization, clearing and settlement) to our financial institution and merchant clients through VisaNet, our proprietary advanced transaction processing network. | What are the primary transaction processing services provided by Visa through VisaNet? |
| Information about legal proceedings is included in Item 8 of the Annual Report on Form 10-K, as referenced in Item 3. | What item in the Annual Report on Form 10-K provides information about legal proceedings? |
| Investing activities used cash of $3.0 billion in 2022. | What was the net cash used by investing activities in 2022? |
* Loss: [MatryoshkaLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters:
```json
{
"loss": "MultipleNegativesRankingLoss",
"matryoshka_dims": [
768,
512,
256,
128,
64
],
"matryoshka_weights": [
1,
1,
1,
1,
1
],
"n_dims_per_step": -1
}
```
### Training Hyperparameters
#### Non-Default Hyperparameters
- `eval_strategy`: epoch
- `per_device_train_batch_size`: 32
- `per_device_eval_batch_size`: 16
- `gradient_accumulation_steps`: 16
- `learning_rate`: 2e-05
- `num_train_epochs`: 4
- `lr_scheduler_type`: cosine
- `warmup_ratio`: 0.1
- `bf16`: True
- `load_best_model_at_end`: True
- `batch_sampler`: no_duplicates
#### All Hyperparameters
Click to expand
- `overwrite_output_dir`: False
- `do_predict`: False
- `eval_strategy`: epoch
- `prediction_loss_only`: True
- `per_device_train_batch_size`: 32
- `per_device_eval_batch_size`: 16
- `per_gpu_train_batch_size`: None
- `per_gpu_eval_batch_size`: None
- `gradient_accumulation_steps`: 16
- `eval_accumulation_steps`: None
- `torch_empty_cache_steps`: None
- `learning_rate`: 2e-05
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `max_grad_norm`: 1.0
- `num_train_epochs`: 4
- `max_steps`: -1
- `lr_scheduler_type`: cosine
- `lr_scheduler_kwargs`: {}
- `warmup_ratio`: 0.1
- `warmup_steps`: 0
- `log_level`: passive
- `log_level_replica`: warning
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `save_safetensors`: True
- `save_on_each_node`: False
- `save_only_model`: False
- `restore_callback_states_from_checkpoint`: False
- `no_cuda`: False
- `use_cpu`: False
- `use_mps_device`: False
- `seed`: 42
- `data_seed`: None
- `jit_mode_eval`: False
- `use_ipex`: False
- `bf16`: True
- `fp16`: False
- `fp16_opt_level`: O1
- `half_precision_backend`: auto
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: None
- `local_rank`: 0
- `ddp_backend`: None
- `tpu_num_cores`: None
- `tpu_metrics_debug`: False
- `debug`: []
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 0
- `dataloader_prefetch_factor`: None
- `past_index`: -1
- `disable_tqdm`: False
- `remove_unused_columns`: True
- `label_names`: None
- `load_best_model_at_end`: True
- `ignore_data_skip`: False
- `fsdp`: []
- `fsdp_min_num_params`: 0
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- `fsdp_transformer_layer_cls_to_wrap`: None
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `deepspeed`: None
- `label_smoothing_factor`: 0.0
- `optim`: adamw_torch
- `optim_args`: None
- `adafactor`: False
- `group_by_length`: False
- `length_column_name`: length
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `skip_memory_metrics`: True
- `use_legacy_prediction_loop`: False
- `push_to_hub`: False
- `resume_from_checkpoint`: None
- `hub_model_id`: None
- `hub_strategy`: every_save
- `hub_private_repo`: None
- `hub_always_push`: False
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `include_inputs_for_metrics`: False
- `include_for_metrics`: []
- `eval_do_concat_batches`: True
- `fp16_backend`: auto
- `push_to_hub_model_id`: None
- `push_to_hub_organization`: None
- `mp_parameters`:
- `auto_find_batch_size`: False
- `full_determinism`: False
- `torchdynamo`: None
- `ray_scope`: last
- `ddp_timeout`: 1800
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `dispatch_batches`: None
- `split_batches`: None
- `include_tokens_per_second`: False
- `include_num_input_tokens_seen`: False
- `neftune_noise_alpha`: None
- `optim_target_modules`: None
- `batch_eval_metrics`: False
- `eval_on_start`: False
- `use_liger_kernel`: False
- `eval_use_gather_object`: False
- `average_tokens_across_devices`: False
- `prompts`: None
- `batch_sampler`: no_duplicates
- `multi_dataset_batch_sampler`: proportional