metadata

language:
  - ja
library_name: sentence-transformers
tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
metrics:
  - pearson_cosine
  - spearman_cosine
  - pearson_manhattan
  - spearman_manhattan
  - pearson_euclidean
  - spearman_euclidean
  - pearson_dot
  - spearman_dot
  - pearson_max
  - spearman_max
widget: []
pipeline_tag: sentence-similarity
datasets:
  - hpprc/emb
  - hpprc/mqa-ja
  - google-research-datasets/paws-x
base_model: pkshatech/GLuCoSE-base-ja

SentenceTransformer

This is a sentence-transformers model trained. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Maximum Sequence Length: 512 tokens
Output Dimensionality: 768 tokens
Similarity Function: Cosine Similarity

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: LukeModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("pkshatech/GLuCoSE-base-ja-v2")
# Run inference
sentences = [
    'The weather is lovely today.',
    "It's so sunny outside!",
    'He drove to the stadium.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Training Details

Framework Versions

Python: 3.10.13
Sentence Transformers: 3.0.0
Transformers: 4.41.2
PyTorch: 2.3.1+cu118
Accelerate: 0.30.1
Datasets: 2.19.2
Tokenizers: 0.19.1

Benchmarks

Zero-shot Search

Evaluated with MIRACL-ja, [JQARA][https://huggingface.co/datasets/hotchpotch/JQaRA] and [MLDR-ja][https://huggingface.co/datasets/Shitao/MLDR].

model	size	MIRACL Recall@5	JQaRA nDCG@10	MLDR nDCG@10
me5-base	0.3B	84.2	47.2	25.4
GLuCoSE	0.1B	53.3	30.8	25.2
GLuCoSE v2	0.1B	85.5	60.6	33.8

JMTEB

Evaluated with [JMTEB][https://github.com/sbintuitions/JMTEB].

Time-consuming [‘amazon_review_classification’, ‘mrtydi’, ‘jaqket’, ‘esci’] were excluded and evaluated.
The average is a macro-average per task.

model	size	Class.	Ret.	STS.	Clus.	Pair.	Avg.
me5-base	0.3B	75.1	80.6	80.5	52.6	62.4	70.2
GLuCoSE	0.1B	82.6	69.8	78.2	51.5	66.2	69.7
GLuCoSE v2	0.1B	80.5	82.8	83.0	49.8	62.4	71.7

pkshatech
/

GLuCoSE-base-ja-v2