ayoubkirouane
commited on
Commit
·
3291976
1
Parent(s):
25ed5a5
Update README.md
Browse files
README.md
CHANGED
@@ -9,3 +9,46 @@ model_name: microsoft/git-base
|
|
9 |
pipeline_tag: image-to-text
|
10 |
library_name: transformers
|
11 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
9 |
pipeline_tag: image-to-text
|
10 |
library_name: transformers
|
11 |
---
|
12 |
+
|
13 |
+
|
14 |
+
# Model Details
|
15 |
+
+ **Model Name**: Git-base-One-Piece
|
16 |
+
+ **Base Model**: Microsoft's "git-base" model
|
17 |
+
+ **Model Type**: Generative Image-to-Text (GIT)
|
18 |
+
+ **Fine-Tuned** On: 'One-Piece-anime-captions' dataset
|
19 |
+
+ **Fine-Tuning Purpose**: To generate text captions for images related to the anime series "One Piece."
|
20 |
+
|
21 |
+
|
22 |
+
|
23 |
+
## Model Description
|
24 |
+
**Git-base-One-Piece** is a fine-tuned variant of Microsoft's **git-base** model, specifically trained for the task of generating descriptive text captions for images from the **One-Piece-anime-captions**'** dataset.
|
25 |
+
|
26 |
+
The dataset consists of **856 {image: caption}** pairs, providing a substantial and diverse training corpus for the model.
|
27 |
+
|
28 |
+
The model is conditioned on both CLIP image tokens and text tokens and employs a **teacher forcing** training approach. It predicts the next text token while considering the context provided by the image and previous text tokens.
|
29 |
+
|
30 |
+
|
31 |
+
## Limitations
|
32 |
+
+ The quality of generated captions may vary depending on the complexity and diversity of images from the 'One-Piece-anime-captions' dataset.
|
33 |
+
+ The model's output is based on the data it was fine-tuned on, so it may not generalize well to images outside the dataset's domain.
|
34 |
+
Generating highly detailed or contextually accurate captions may still be a challenge.
|
35 |
+
|
36 |
+
|
37 |
+
## Usage
|
38 |
+
|
39 |
+
```python
|
40 |
+
# Use a pipeline as a high-level helper
|
41 |
+
from transformers import pipeline
|
42 |
+
|
43 |
+
pipe = pipeline("image-to-text", model="ayoubkirouane/git-base-One-Piece")
|
44 |
+
```
|
45 |
+
|
46 |
+
**or**
|
47 |
+
|
48 |
+
```python
|
49 |
+
# Load model directly
|
50 |
+
from transformers import AutoProcessor, AutoModelForCausalLM
|
51 |
+
|
52 |
+
processor = AutoProcessor.from_pretrained("ayoubkirouane/git-base-One-Piece")
|
53 |
+
model = AutoModelForCausalLM.from_pretrained("ayoubkirouane/git-base-One-Piece")
|
54 |
+
```
|