|
--- |
|
license: afl-3.0 |
|
--- |
|
|
|
|
|
This model was trained with the [Rankers Library](`https://github.com/davidmrau/rankers`) please check https://github.com/davidmrau/rankers#Evaluation to learn how to use the model. |
|
|
|
This Cross Encoder Ranking Model is invariant to the input order by removing the position embeddings (by being set to zero) during fine-tuning. It was trained on the official MS MARCO training triples using the following training parameters: |
|
|
|
- batch size: 128 |
|
|
|
- learning rate 3e-6 |
|
|
|
- Optimizer: Adam |
|
|
|
|
|
<strong>Example on how to load and use BOW-BERT: <strong> |
|
```python |
|
from transformers import AutoModelForSequenceClassification, AutoTokenizer |
|
|
|
# load model |
|
model = AutoModelForSequenceClassification.from_pretrained('dmrau/bow-bert') |
|
# load tokenizer |
|
tokenizer = AutoTokenizer.from_pretrained('bert-base-uncased') |
|
|
|
# tokenize query and passage and concatenate them |
|
inp = tokenizer(['this is a query','query a is this'], ['this is a passage', 'passage a is this'], return_tensors='pt') |
|
# get estimated score |
|
print('score', model(**inp).logits[:, 1]) |
|
|
|
### outputs identical scores for different |
|
### word orders as the model is order invariant: |
|
# scores: [-2.9463, -2.9463] |
|
``` |
|
|
|
|
|
|
|
<strong> Cite us:<strong> |
|
|
|
|
|
``` |
|
@article{rau2022role, |
|
title={The Role of Complex NLP in Transformers for Text Ranking?}, |
|
author={Rau, David and Kamps, Jaap}, |
|
journal={arXiv preprint arXiv:2207.02522}, |
|
year={2022} |
|
} |
|
``` |
|
|
