Hugging Face's logo

ldldld
/
snowflake-arctic-embed-m-finetuned

Sentence Similarity
sentence-transformers
Safetensors
bert
feature-extraction
Generated from Trainer
dataset_size:600
loss:MatryoshkaLoss
loss:MultipleNegativesRankingLoss
Eval Results
text-embeddings-inference
Inference Endpoints
Model card Files Community
snowflake-arctic-embed-m-finetuned / README.md
ldldld's picture
ldldld
Add new SentenceTransformer model.
086cd7c verified
preview code
|
raw
history blame contribute delete
37.3 kB
---
base_model: Snowflake/snowflake-arctic-embed-m
library_name: sentence-transformers
metrics:
- cosine_accuracy@1
- cosine_accuracy@3
- cosine_accuracy@5
- cosine_accuracy@10
- cosine_precision@1
- cosine_precision@3
- cosine_precision@5
- cosine_precision@10
- cosine_recall@1
- cosine_recall@3
- cosine_recall@5
- cosine_recall@10
- cosine_ndcg@10
- cosine_mrr@10
- cosine_map@100
- dot_accuracy@1
- dot_accuracy@3
- dot_accuracy@5
- dot_accuracy@10
- dot_precision@1
- dot_precision@3
- dot_precision@5
- dot_precision@10
- dot_recall@1
- dot_recall@3
- dot_recall@5
- dot_recall@10
- dot_ndcg@10
- dot_mrr@10
- dot_map@100
pipeline_tag: sentence-similarity
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:600
- loss:MatryoshkaLoss
- loss:MultipleNegativesRankingLoss
widget:
- source_sentence: What is the purpose of the Artificial Intelligence Ethics for the
Intelligence Community as mentioned in the context?
sentences:
- "You should be able to opt out, where appropriate, and \nhave access to a person\
\ who can quickly consider and \nremedy problems you encounter. You should be\
\ able to opt \nout from automated systems in favor of a human alternative, where\
\ \nappropriate. Appropriateness should be determined based on rea­\nsonable expectations\
\ in a given context and with a focus on ensuring \nbroad accessibility and protecting\
\ the public from especially harm­\nful impacts. In some cases, a human or other\
\ alternative may be re­\nquired by law. You should have access to timely human\
\ consider­\nation and remedy by a fallback and escalation process if an automat­\n\
ed system fails, it produces an error, or you would like to appeal or \ncontest\
\ its impacts on you. Human consideration and fallback \nshould be accessible,\
\ equitable, effective, maintained, accompanied \nby appropriate operator training,\
\ and should not impose an unrea­\nsonable burden on the public. Automated systems\
\ with an intended"
- "points to numerous examples of effective and proactive stakeholder engagement,\
\ including the Community-\nBased Participatory Research Program developed by\
\ the National Institutes of Health and the participatory \ntechnology assessments\
\ developed by the National Oceanic and Atmospheric Administration.18\nThe National\
\ Institute of Standards and Technology (NIST) is developing a risk \nmanagement\
\ framework to better manage risks posed to individuals, organizations, and \n\
society by AI.19 The NIST AI Risk Management Framework, as mandated by Congress,\
\ is intended for \nvoluntary use to help incorporate trustworthiness considerations\
\ into the design, development, use, and \nevaluation of AI products, services,\
\ and systems. The NIST framework is being developed through a consensus-\ndriven,\
\ open, transparent, and collaborative process that includes workshops and other\
\ opportunities to provide \ninput. The NIST framework aims to foster the development\
\ of innovative approaches to address"
- "of Artificial Intelligence Ethics for the Intelligence Community to guide personnel\
\ on whether and how to \ndevelop and use AI in furtherance of the IC's mission,\
\ as well as an AI Ethics Framework to help implement \nthese principles.22\n\
The National Science Foundation (NSF) funds extensive research to help foster\
\ the \ndevelopment of automated systems that adhere to and advance their safety,\
\ security and \neffectiveness. Multiple NSF programs support research that directly\
\ addresses many of these principles: \nthe National AI Research Institutes23\
\ support research on all aspects of safe, trustworthy, fair, and explainable\
\ \nAI algorithms and systems; the Cyber Physical Systems24 program supports research\
\ on developing safe \nautonomous and cyber physical systems with AI components;\
\ the Secure and Trustworthy Cyberspace25 \nprogram supports research on cybersecurity\
\ and privacy enhancing technologies in automated systems; the"
- source_sentence: How does the Department of Defense's approach to AI ethics differ
from that of the Department of Energy?
sentences:
- "NOTICE & \nEXPLANATION \nWHAT SHOULD BE EXPECTED OF AUTOMATED SYSTEMS\nThe expectations\
\ for automated systems are meant to serve as a blueprint for the development\
\ of additional \ntechnical standards and practices that are tailored for particular\
\ sectors and contexts. \nTailored to the level of risk. An assessment should\
\ be done to determine the level of risk of the auto­\nmated system. In settings\
\ where the consequences are high as determined by a risk assessment, or extensive\
\ \noversight is expected (e.g., in criminal justice or some public sector settings),\
\ explanatory mechanisms should \nbe built into the system design so that the\
\ system’s full behavior can be explained in advance (i.e., only fully \ntransparent\
\ models should be used), rather than as an after-the-decision interpretation.\
\ In other settings, the \nextent of explanation provided should be tailored to\
\ the risk level."
- "SAFE AND EFFECTIVE \nSYSTEMS \nHOW THESE PRINCIPLES CAN MOVE INTO PRACTICE\n\
Real-life examples of how these principles can become reality, through laws, policies,\
\ and practical \ntechnical and sociotechnical approaches to protecting rights,\
\ opportunities, and access. ­\nSome U.S government agencies have developed specific\
\ frameworks for ethical use of AI \nsystems. The Department of Energy (DOE) has\
\ activated the AI Advancement Council that oversees coordina-\ntion and advises\
\ on implementation of the DOE AI Strategy and addresses issues and/or escalations\
\ on the \nethical use and development of AI systems.20 The Department of Defense\
\ has adopted Artificial Intelligence \nEthical Principles, and tenets for Responsible\
\ Artificial Intelligence specifically tailored to its national \nsecurity and\
\ defense activities.21 Similarly, the U.S. Intelligence Community (IC) has developed\
\ the Principles"
- "Formal Methods in the Field26 program supports research on rigorous formal verification\
\ and analysis of \nautomated systems and machine learning, and the Designing\
\ Accountable Software Systems27 program supports \nresearch on rigorous and reproducible\
\ methodologies for developing software systems with legal and regulatory \ncompliance\
\ in mind. \nSome state legislatures have placed strong transparency and validity\
\ requirements on \nthe use of pretrial risk assessments. The use of algorithmic\
\ pretrial risk assessments has been a \ncause of concern for civil rights groups.28\
\ Idaho Code Section 19-1910, enacted in 2019,29 requires that any \npretrial\
\ risk assessment, before use in the state, first be \"shown to be free of bias\
\ against any class of \nindividuals protected from discrimination by state or\
\ federal law\", that any locality using a pretrial risk \nassessment must first\
\ formally validate the claim of its being free of bias, that \"all documents,\
\ records, and"
- source_sentence: What are the expectations for automated systems intended to serve
as a blueprint for?
sentences:
- "help to mitigate biases and potential harms. \nGuarding against proxies. Directly\
\ using demographic information in the design, development, or \ndeployment of\
\ an automated system (for purposes other than evaluating a system for discrimination\
\ or using \na system to counter discrimination) runs a high risk of leading to\
\ algorithmic discrimination and should be \navoided. In many cases, attributes\
\ that are highly correlated with demographic features, known as proxies, can\
\ \ncontribute to algorithmic discrimination. In cases where use of the demographic\
\ features themselves would \nlead to illegal algorithmic discrimination, reliance\
\ on such proxies in decision-making (such as that facilitated \nby an algorithm)\
\ may also be prohibited by law. Proactive testing should be performed to identify\
\ proxies by \ntesting for correlation between demographic information and attributes\
\ in any data used as part of system"
- "describes three broad challenges for mitigating bias – datasets, testing and\
\ evaluation, and human factors – and \nintroduces preliminary guidance for addressing\
\ them. Throughout, the special publication takes a socio-\ntechnical perspective\
\ to identifying and managing AI bias. \n29\nAlgorithmic \nDiscrimination \nProtections"
- "SAFE AND EFFECTIVE \nSYSTEMS \nWHAT SHOULD BE EXPECTED OF AUTOMATED SYSTEMS\n\
The expectations for automated systems are meant to serve as a blueprint for the\
\ development of additional \ntechnical standards and practices that are tailored\
\ for particular sectors and contexts. \nDerived data sources tracked and reviewed\
\ carefully. Data that is derived from other data through \nthe use of algorithms,\
\ such as data derived or inferred from prior model outputs, should be identified\
\ and \ntracked, e.g., via a specialized type in a data schema. Derived data should\
\ be viewed as potentially high-risk \ninputs that may lead to feedback loops,\
\ compounded harm, or inaccurate results. Such sources should be care­\nfully\
\ validated against the risk of collateral consequences. \nData reuse limits in\
\ sensitive domains. Data reuse, and especially data reuse in a new context, can\
\ result \nin the spreading and scaling of harms. Data from some domains, including\
\ criminal justice data and data indi­"
- source_sentence: What should individuals have access to regarding their data decisions
and the impact of surveillance technologies?
sentences:
- '•
Searches for “Black girls,” “Asian girls,” or “Latina girls” return predominantly39
sexualized content, rather
than role models, toys, or activities.40 Some search engines have been working
to reduce the prevalence of
these results, but the problem remains.41
•
Advertisement delivery systems that predict who is most likely to click on a job
advertisement end up deliv-
ering ads in ways that reinforce racial and gender stereotypes, such as overwhelmingly
directing supermar-
ket cashier ads to women and jobs with taxi companies to primarily Black people.42­
•
Body scanners, used by TSA at airport checkpoints, require the operator to select
a “male” or “female”
scanning setting based on the passenger’s sex, but the setting is chosen based
on the operator’s perception of
the passenger’s gender identity. These scanners are more likely to flag transgender
travelers as requiring
extra screening done by a person. Transgender travelers have described degrading
experiences associated'
- "information used to build or validate the risk assessment shall be open to public\
\ inspection,\" and that assertions \nof trade secrets cannot be used \"to quash\
\ discovery in a criminal matter by a party to a criminal case.\" \n22"
- "tect privacy and civil liberties. Continuous surveillance and monitoring \nshould\
\ not be used in education, work, housing, or in other contexts where the \nuse\
\ of such surveillance technologies is likely to limit rights, opportunities,\
\ or \naccess. Whenever possible, you should have access to reporting that confirms\
\ \nyour data decisions have been respected and provides an assessment of the\
\ \npotential impact of surveillance technologies on your rights, opportunities,\
\ or \naccess. \nDATA PRIVACY\n30"
- source_sentence: What are the implications of the digital divide highlighted in
Andrew Kenney's article regarding unemployment benefits?
sentences:
- "cating adverse outcomes in domains such as finance, employment, and housing,\
\ is especially sensitive, and in \nsome cases its reuse is limited by law. Accordingly,\
\ such data should be subject to extra oversight to ensure \nsafety and efficacy.\
\ Data reuse of sensitive domain data in other contexts (e.g., criminal data reuse\
\ for civil legal \nmatters or private sector use) should only occur where use\
\ of such data is legally authorized and, after examina­\ntion, has benefits for\
\ those impacted by the system that outweigh identified risks and, as appropriate,\
\ reason­\nable measures have been implemented to mitigate the identified risks.\
\ Such data should be clearly labeled to \nidentify contexts for limited reuse\
\ based on sensitivity. Where possible, aggregated datasets may be useful for\
\ \nreplacing individual-level sensitive data. \nDemonstrate the safety and effectiveness\
\ of the system \nIndependent evaluation. Automated systems should be designed\
\ to allow for independent evaluation (e.g.,"
- "5. Environmental Impacts: Impacts due to high compute resource utilization in\
\ training or \noperating GAI models, and related outcomes that may adversely\
\ impact ecosystems. \n6. Harmful Bias or Homogenization: Amplification and exacerbation\
\ of historical, societal, and \nsystemic biases; performance disparities8 between\
\ sub-groups or languages, possibly due to \nnon-representative training data,\
\ that result in discrimination, amplification of biases, or \nincorrect presumptions\
\ about performance; undesired homogeneity that skews system or model \noutputs,\
\ which may be erroneous, lead to ill-founded decision-making, or amplify harmful\
\ \nbiases. \n7. Human-AI Configuration: Arrangements of or interactions between\
\ a human and an AI system \nwhich can result in the human inappropriately anthropomorphizing\
\ GAI systems or experiencing \nalgorithmic aversion, automation bias, over-reliance,\
\ or emotional entanglement with GAI \nsystems."
- 'https://bipartisanpolicy.org/blog/the-low-down-on-ballot-curing/
101. Andrew Kenney. ''I''m shocked that they need to have a smartphone'': System
for unemployment
benefits exposes digital divide. USA Today. May 2, 2021.
https://www.usatoday.com/story/tech/news/2021/05/02/unemployment-benefits-system-leaving­
people-behind/4915248001/
102. Allie Gross. UIA lawsuit shows how the state criminalizes the unemployed.
Detroit Metro-Times.
Sep. 18, 2015.
https://www.metrotimes.com/news/uia-lawsuit-shows-how-the-state-criminalizes-the­
unemployed-2369412
103. Maia Szalavitz. The Pain Was Unbearable. So Why Did Doctors Turn Her Away?
Wired. Aug. 11,
2021. https://www.wired.com/story/opioid-drug-addiction-algorithm-chronic-pain/
104. Spencer Soper. Fired by Bot at Amazon: "It''s You Against the Machine". Bloomberg,
Jun. 28, 2021.
https://www.bloomberg.com/news/features/2021-06-28/fired-by-bot-amazon-turns-to-machine­
managers-and-workers-are-losing-out'
model-index:
- name: SentenceTransformer based on Snowflake/snowflake-arctic-embed-m
results:
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: Unknown
type: unknown
metrics:
- type: cosine_accuracy@1
value: 0.73
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.9
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.935
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.96
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.73
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.3
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.187
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.096
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.73
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.9
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.935
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.96
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.8511693160760204
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.8155396825396827
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.8172228277187864
name: Cosine Map@100
- type: dot_accuracy@1
value: 0.73
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.9
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.935
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.96
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.73
name: Dot Precision@1
- type: dot_precision@3
value: 0.3
name: Dot Precision@3
- type: dot_precision@5
value: 0.187
name: Dot Precision@5
- type: dot_precision@10
value: 0.096
name: Dot Precision@10
- type: dot_recall@1
value: 0.73
name: Dot Recall@1
- type: dot_recall@3
value: 0.9
name: Dot Recall@3
- type: dot_recall@5
value: 0.935
name: Dot Recall@5
- type: dot_recall@10
value: 0.96
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.8511693160760204
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.8155396825396827
name: Dot Mrr@10
- type: dot_map@100
value: 0.8172228277187864
name: Dot Map@100
---
# SentenceTransformer based on Snowflake/snowflake-arctic-embed-m
This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [Snowflake/snowflake-arctic-embed-m](https://huggingface.co/Snowflake/snowflake-arctic-embed-m). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
## Model Details
### Model Description
- **Model Type:** Sentence Transformer
- **Base model:** [Snowflake/snowflake-arctic-embed-m](https://huggingface.co/Snowflake/snowflake-arctic-embed-m) <!-- at revision e2b128b9fa60c82b4585512b33e1544224ffff42 -->
- **Maximum Sequence Length:** 512 tokens
- **Output Dimensionality:** 768 tokens
- **Similarity Function:** Cosine Similarity
<!-- - **Training Dataset:** Unknown -->
<!-- - **Language:** Unknown -->
<!-- - **License:** Unknown -->
### Model Sources
- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
### Full Model Architecture
```
SentenceTransformer(
(0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel
(1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
(2): Normalize()
)
```
## Usage
### Direct Usage (Sentence Transformers)
First install the Sentence Transformers library:
```bash
pip install -U sentence-transformers
```
Then you can load this model and run inference.
```python
from sentence_transformers import SentenceTransformer
# Download from the 🤗 Hub
model = SentenceTransformer("ldldld/snowflake-arctic-embed-m-finetuned")
# Run inference
sentences = [
"What are the implications of the digital divide highlighted in Andrew Kenney's article regarding unemployment benefits?",
'https://bipartisanpolicy.org/blog/the-low-down-on-ballot-curing/\n101. Andrew Kenney. \'I\'m shocked that they need to have a smartphone\': System for unemployment\nbenefits exposes digital divide. USA Today. May 2, 2021.\nhttps://www.usatoday.com/story/tech/news/2021/05/02/unemployment-benefits-system-leaving\xad\npeople-behind/4915248001/\n102. Allie Gross. UIA lawsuit shows how the state criminalizes the unemployed. Detroit Metro-Times.\nSep. 18, 2015.\nhttps://www.metrotimes.com/news/uia-lawsuit-shows-how-the-state-criminalizes-the\xad\nunemployed-2369412\n103. Maia Szalavitz. The Pain Was Unbearable. So Why Did Doctors Turn Her Away? Wired. Aug. 11,\n2021. https://www.wired.com/story/opioid-drug-addiction-algorithm-chronic-pain/\n104. Spencer Soper. Fired by Bot at Amazon: "It\'s You Against the Machine". Bloomberg, Jun. 28, 2021.\nhttps://www.bloomberg.com/news/features/2021-06-28/fired-by-bot-amazon-turns-to-machine\xad\nmanagers-and-workers-are-losing-out',
'5. Environmental Impacts: Impacts due to high compute resource utilization in training or \noperating GAI models, and related outcomes that may adversely impact ecosystems. \n6. Harmful Bias or Homogenization: Amplification and exacerbation of historical, societal, and \nsystemic biases; performance disparities8 between sub-groups or languages, possibly due to \nnon-representative training data, that result in discrimination, amplification of biases, or \nincorrect presumptions about performance; undesired homogeneity that skews system or model \noutputs, which may be erroneous, lead to ill-founded decision-making, or amplify harmful \nbiases. \n7. Human-AI Configuration: Arrangements of or interactions between a human and an AI system \nwhich can result in the human inappropriately anthropomorphizing GAI systems or experiencing \nalgorithmic aversion, automation bias, over-reliance, or emotional entanglement with GAI \nsystems.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]
# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
```
<!--
### Direct Usage (Transformers)
<details><summary>Click to see the direct usage in Transformers</summary>
</details>
-->
<!--
### Downstream Usage (Sentence Transformers)
You can finetune this model on your own dataset.
<details><summary>Click to expand</summary>
</details>
-->
<!--
### Out-of-Scope Use
*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->
## Evaluation
### Metrics
#### Information Retrieval
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)
| Metric | Value |
|:--------------------|:-----------|
| cosine_accuracy@1 | 0.73 |
| cosine_accuracy@3 | 0.9 |
| cosine_accuracy@5 | 0.935 |
| cosine_accuracy@10 | 0.96 |
| cosine_precision@1 | 0.73 |
| cosine_precision@3 | 0.3 |
| cosine_precision@5 | 0.187 |
| cosine_precision@10 | 0.096 |
| cosine_recall@1 | 0.73 |
| cosine_recall@3 | 0.9 |
| cosine_recall@5 | 0.935 |
| cosine_recall@10 | 0.96 |
| cosine_ndcg@10 | 0.8512 |
| cosine_mrr@10 | 0.8155 |
| **cosine_map@100** | **0.8172** |
| dot_accuracy@1 | 0.73 |
| dot_accuracy@3 | 0.9 |
| dot_accuracy@5 | 0.935 |
| dot_accuracy@10 | 0.96 |
| dot_precision@1 | 0.73 |
| dot_precision@3 | 0.3 |
| dot_precision@5 | 0.187 |
| dot_precision@10 | 0.096 |
| dot_recall@1 | 0.73 |
| dot_recall@3 | 0.9 |
| dot_recall@5 | 0.935 |
| dot_recall@10 | 0.96 |
| dot_ndcg@10 | 0.8512 |
| dot_mrr@10 | 0.8155 |
| dot_map@100 | 0.8172 |
<!--
## Bias, Risks and Limitations
*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->
<!--
### Recommendations
*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->
## Training Details
### Training Dataset
#### Unnamed Dataset
* Size: 600 training samples
* Columns: <code>sentence_0</code> and <code>sentence_1</code>
* Approximate statistics based on the first 600 samples:
| | sentence_0 | sentence_1 |
|:--------|:-----------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------|
| type | string | string |
| details | <ul><li>min: 12 tokens</li><li>mean: 20.66 tokens</li><li>max: 34 tokens</li></ul> | <ul><li>min: 21 tokens</li><li>mean: 165.88 tokens</li><li>max: 512 tokens</li></ul> |
* Samples:
| sentence_0 | sentence_1 |
|:------------------------------------------------------------------------------------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| <code>What is the main purpose of the "Blueprint for an AI Bill of Rights" as indicated in the context?</code> | <code>BLUEPRINT FOR AN <br>AI BILL OF <br>RIGHTS <br>MAKING AUTOMATED <br>SYSTEMS WORK FOR <br>THE AMERICAN PEOPLE <br>OCTOBER 2022</code> |
| <code>When was the "Blueprint for an AI Bill of Rights" created?</code> | <code>BLUEPRINT FOR AN <br>AI BILL OF <br>RIGHTS <br>MAKING AUTOMATED <br>SYSTEMS WORK FOR <br>THE AMERICAN PEOPLE <br>OCTOBER 2022</code> |
| <code>What was the purpose of the Blueprint for an AI Bill of Rights published by the White House Office of Science and Technology Policy in October 2022?</code> | <code>About this Document <br>The Blueprint for an AI Bill of Rights: Making Automated Systems Work for the American People was <br>published by the White House Office of Science and Technology Policy in October 2022. This framework was <br>released one year after OSTP announced the launch of a process to develop “a bill of rights for an AI-powered <br>world.” Its release follows a year of public engagement to inform this initiative. The framework is available <br>online at: https://www.whitehouse.gov/ostp/ai-bill-of-rights <br>About the Office of Science and Technology Policy <br>The Office of Science and Technology Policy (OSTP) was established by the National Science and Technology <br>Policy, Organization, and Priorities Act of 1976 to provide the President and others within the Executive Office <br>of the President with advice on the scientific, engineering, and technological aspects of the economy, national</code> |
* Loss: [<code>MatryoshkaLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters:
```json
{
"loss": "MultipleNegativesRankingLoss",
"matryoshka_dims": [
768,
512,
256,
128,
64
],
"matryoshka_weights": [
1,
1,
1,
1,
1
],
"n_dims_per_step": -1
}
```
### Training Hyperparameters
#### Non-Default Hyperparameters
- `eval_strategy`: steps
- `per_device_train_batch_size`: 20
- `per_device_eval_batch_size`: 20
- `num_train_epochs`: 5
- `multi_dataset_batch_sampler`: round_robin
#### All Hyperparameters
<details><summary>Click to expand</summary>
- `overwrite_output_dir`: False
- `do_predict`: False
- `eval_strategy`: steps
- `prediction_loss_only`: True
- `per_device_train_batch_size`: 20
- `per_device_eval_batch_size`: 20
- `per_gpu_train_batch_size`: None
- `per_gpu_eval_batch_size`: None
- `gradient_accumulation_steps`: 1
- `eval_accumulation_steps`: None
- `torch_empty_cache_steps`: None
- `learning_rate`: 5e-05
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `max_grad_norm`: 1
- `num_train_epochs`: 5
- `max_steps`: -1
- `lr_scheduler_type`: linear
- `lr_scheduler_kwargs`: {}
- `warmup_ratio`: 0.0
- `warmup_steps`: 0
- `log_level`: passive
- `log_level_replica`: warning
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `save_safetensors`: True
- `save_on_each_node`: False
- `save_only_model`: False
- `restore_callback_states_from_checkpoint`: False
- `no_cuda`: False
- `use_cpu`: False
- `use_mps_device`: False
- `seed`: 42
- `data_seed`: None
- `jit_mode_eval`: False
- `use_ipex`: False
- `bf16`: False
- `fp16`: False
- `fp16_opt_level`: O1
- `half_precision_backend`: auto
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: None
- `local_rank`: 0
- `ddp_backend`: None
- `tpu_num_cores`: None
- `tpu_metrics_debug`: False
- `debug`: []
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 0
- `dataloader_prefetch_factor`: None
- `past_index`: -1
- `disable_tqdm`: False
- `remove_unused_columns`: True
- `label_names`: None
- `load_best_model_at_end`: False
- `ignore_data_skip`: False
- `fsdp`: []
- `fsdp_min_num_params`: 0
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- `fsdp_transformer_layer_cls_to_wrap`: None
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `deepspeed`: None
- `label_smoothing_factor`: 0.0
- `optim`: adamw_torch
- `optim_args`: None
- `adafactor`: False
- `group_by_length`: False
- `length_column_name`: length
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `skip_memory_metrics`: True
- `use_legacy_prediction_loop`: False
- `push_to_hub`: False
- `resume_from_checkpoint`: None
- `hub_model_id`: None
- `hub_strategy`: every_save
- `hub_private_repo`: False
- `hub_always_push`: False
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `include_inputs_for_metrics`: False
- `eval_do_concat_batches`: True
- `fp16_backend`: auto
- `push_to_hub_model_id`: None
- `push_to_hub_organization`: None
- `mp_parameters`:
- `auto_find_batch_size`: False
- `full_determinism`: False
- `torchdynamo`: None
- `ray_scope`: last
- `ddp_timeout`: 1800
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `dispatch_batches`: None
- `split_batches`: None
- `include_tokens_per_second`: False
- `include_num_input_tokens_seen`: False
- `neftune_noise_alpha`: None
- `optim_target_modules`: None
- `batch_eval_metrics`: False
- `eval_on_start`: False
- `eval_use_gather_object`: False
- `batch_sampler`: batch_sampler
- `multi_dataset_batch_sampler`: round_robin
</details>
### Training Logs
| Epoch | Step | cosine_map@100 |
|:------:|:----:|:--------------:|
| 1.0 | 30 | 0.7953 |
| 1.6667 | 50 | 0.8326 |
| 2.0 | 60 | 0.8277 |
| 3.0 | 90 | 0.8250 |
| 3.3333 | 100 | 0.8284 |
| 4.0 | 120 | 0.8200 |
| 5.0 | 150 | 0.8172 |
### Framework Versions
- Python: 3.10.12
- Sentence Transformers: 3.1.1
- Transformers: 4.44.2
- PyTorch: 2.4.1+cu121
- Accelerate: 0.34.2
- Datasets: 3.0.0
- Tokenizers: 0.19.1
## Citation
### BibTeX
#### Sentence Transformers
```bibtex
@inproceedings{reimers-2019-sentence-bert,
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
author = "Reimers, Nils and Gurevych, Iryna",
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
month = "11",
year = "2019",
publisher = "Association for Computational Linguistics",
url = "https://arxiv.org/abs/1908.10084",
}
```
#### MatryoshkaLoss
```bibtex
@misc{kusupati2024matryoshka,
title={Matryoshka Representation Learning},
author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
year={2024},
eprint={2205.13147},
archivePrefix={arXiv},
primaryClass={cs.LG}
}
```
#### MultipleNegativesRankingLoss
```bibtex
@misc{henderson2017efficient,
title={Efficient Natural Language Response Suggestion for Smart Reply},
author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
year={2017},
eprint={1705.00652},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
```
<!--
## Glossary
*Clearly define terms in order to be accessible across audiences.*
-->
<!--
## Model Card Authors
*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->
<!--
## Model Card Contact
*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->