Add README

README.md

@@ -0,0 +1,109 @@
+---
+license: apache-2.0
+language: ti
+library_name: sentence-transformers
+pipeline_tag: sentence-similarity
+tags:
+- sentence-transformers
+- feature-extraction
+- sentence-similarity
+- transformers
+widget:
+- text: "ግራፋይት ኣብ መላእ ዓለም ዳርጋ ብምዕሩይ ዝርጋሐ’ዩ ዝርከብ"
+---
+
+# TiELECTRA BiEncoder Model
+
+This model is a [sentence-transformers](https://www.SBERT.net) model for the Tigrinya language based on [TiELECTRA-small](https://huggingface.co/fgaim/tielectra-bi-encoder).
+The maps sentences & paragraphs to a 256 dimensional dense vector space and can be used for tasks like clustering or semantic search.
+
+## Using Sentence-Transformers
+
+Using this model becomes easy when you have sentence-transformersinstalled:
+
+```shell
+pip install -U sentence-transformers
+```
+
+Then use the model as follows:
+
+```python
+from sentence_transformers import SentenceTransformer
+sentences = ["ሓደ ሰብኣይ ፈረስ ይጋልብ ኣሎ።", "ሓንቲ ጓል ክራር ትጻወት ኣላ።"]
+
+model = SentenceTransformer('fgaim/tielectra-bi-encoder')
+embeddings = model.encode(sentences)
+print(embeddings)
+```
+
+## Using 🤗 Transformers
+
+Use the transformers library as follows:
+Pass the input through the transformer model, then you have to apply the right pooling-operation on-top of the contextualized word embeddings.
+
+```python
+import torch
+from transformers import AutoModel, AutoTokenizer
+
+
+# Mean Pooling - Take attention mask into account for correct averaging
+def mean_pooling(model_output, attention_mask):
+    token_embeddings = model_output[0]  # First element of model_output contains all token embeddings
+    input_mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float()
+    return torch.sum(token_embeddings * input_mask_expanded, 1) / torch.clamp(input_mask_expanded.sum(1), min=1e-9)
+
+
+# Sentences we want sentence embeddings for
+sentences = ["ሓደ ሰብኣይ ፈረስ ይጋልብ ኣሎ።", "ሓንቲ ጓል ክራር ትጻወት ኣላ።"]
+
+# Load model from HuggingFace Hub
+tokenizer = AutoTokenizer.from_pretrained("fgaim/tielectra-bi-encoder")
+model = AutoModel.from_pretrained("fgaim/tielectra-bi-encoder")
+
+# Tokenize sentences
+encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors="pt")
+
+# Compute token embeddings
+with torch.no_grad():
+    model_output = model(**encoded_input)
+
+# Perform pooling. In this case, mean pooling.
+sentence_embeddings = mean_pooling(model_output, encoded_input["attention_mask"])
+
+print("Sentence embeddings:", sentence_embeddings)
+```
+
+## Architecture
+
+### Base Model
+
+The model properties:
+| Model Size | Layers  | Attn. Heads | Hidden Size  | FFN  | Parameters  | Max. Seq  |
+|------------|----|----|-----|------|------|------|
+| SMALL      | 12 | 4  | 256 | 1024 | 14M  | 512  |
+
+### BiEncoder Model
+
+- Max Seq Length: `512`
+- Word embedding dimension: `256`
+
+```
+SentenceTransformer(
+  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: ElectraModel
+
+  (1): Pooling({'word_embedding_dimension': 256, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
+)
+```
+
+## Cite
+
+If you use this model in your product or research, you can cite it as follows:
+
+```bibtex
+@article{Fitsum2021TiPLMs,
+  author={Fitsum Gaim and Wonsuk Yang and Jong C. Park},
+  title={Monolingual Pre-trained Language Models for Tigrinya},
+  year=2021,
+  publisher={WiNLP 2021 co-located EMNLP 2021}
+}
+```