Model Card for ModernBERT Fine-Tuned on Social Media Sentiment Analysis

This model is a fine-tuned version of ModernBERT-base tailored for sentiment analysis on social media data. ModernBERT is a modernized bidirectional encoder-only Transformer model pre-trained on 2 trillion tokens of English and code data, with a native context length of up to 8,192 tokens. :contentReference[oaicite:0]{index=0}

Model Details

Model Description

This model is designed to perform sentiment analysis on social media text, classifying posts into positive, negative, or neutral sentiments. It leverages the advanced architecture of ModernBERT, which incorporates recent innovations in Transformer models to enhance performance and efficiency. :contentReference[oaicite:1]{index=1}

Developed by: Chukwuebuka Ezeokeke
Model type: Encoder-only Transformer
Language(s): English
License: Apache 2.0
Fine-tuned from model: ModernBERT-base

Model Sources

Repository: My Implementation
Paper: Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference

Uses

Direct Use

This model can be directly used for sentiment analysis of English-language social media posts, aiding in understanding public opinion, monitoring brand sentiment, and analyzing user feedback.

Downstream Use

The model can be integrated into larger systems for tasks such as:

Customer Feedback Analysis: Automating the analysis of customer sentiments from social media platforms.
Market Research: Gauging public reaction to products or events.
Content Moderation: Identifying potentially harmful or negative content.

Out-of-Scope Use

The model may not perform well on non-English text or on text that deviates significantly from social media language patterns. It is not designed for tasks outside sentiment analysis, such as topic modeling or language translation.

Bias, Risks, and Limitations

While the model aims to provide accurate sentiment analysis, it may inherit biases present in the training data, especially those prevalent in social media language. Users should be cautious when deploying the model in sensitive applications and consider the potential for misclassification.

Recommendations

Bias Mitigation: Regularly assess and mitigate biases by updating the training data and fine-tuning the model as needed.
Performance Monitoring: Continuously monitor the model's performance, especially when applied to new or evolving social media platforms.

How to Get Started with the Model

To use this model, you can load it with the Hugging Face transformers library:

from transformers import AutoModelForSequenceClassification, AutoTokenizer

model_name = "EbukaGaus/EbukaMBert"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

# Example usage
inputs = tokenizer("I love using this new app!", return_tensors="pt")
outputs = model(**inputs)