ayoubkirouane commited on
Commit
3291976
·
1 Parent(s): 25ed5a5

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +43 -0
README.md CHANGED
@@ -9,3 +9,46 @@ model_name: microsoft/git-base
9
  pipeline_tag: image-to-text
10
  library_name: transformers
11
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
9
  pipeline_tag: image-to-text
10
  library_name: transformers
11
  ---
12
+
13
+
14
+ # Model Details
15
+ + **Model Name**: Git-base-One-Piece
16
+ + **Base Model**: Microsoft's "git-base" model
17
+ + **Model Type**: Generative Image-to-Text (GIT)
18
+ + **Fine-Tuned** On: 'One-Piece-anime-captions' dataset
19
+ + **Fine-Tuning Purpose**: To generate text captions for images related to the anime series "One Piece."
20
+
21
+
22
+
23
+ ## Model Description
24
+ **Git-base-One-Piece** is a fine-tuned variant of Microsoft's **git-base** model, specifically trained for the task of generating descriptive text captions for images from the **One-Piece-anime-captions**'** dataset.
25
+
26
+ The dataset consists of **856 {image: caption}** pairs, providing a substantial and diverse training corpus for the model.
27
+
28
+ The model is conditioned on both CLIP image tokens and text tokens and employs a **teacher forcing** training approach. It predicts the next text token while considering the context provided by the image and previous text tokens.
29
+
30
+
31
+ ## Limitations
32
+ + The quality of generated captions may vary depending on the complexity and diversity of images from the 'One-Piece-anime-captions' dataset.
33
+ + The model's output is based on the data it was fine-tuned on, so it may not generalize well to images outside the dataset's domain.
34
+ Generating highly detailed or contextually accurate captions may still be a challenge.
35
+
36
+
37
+ ## Usage
38
+
39
+ ```python
40
+ # Use a pipeline as a high-level helper
41
+ from transformers import pipeline
42
+
43
+ pipe = pipeline("image-to-text", model="ayoubkirouane/git-base-One-Piece")
44
+ ```
45
+
46
+ **or**
47
+
48
+ ```python
49
+ # Load model directly
50
+ from transformers import AutoProcessor, AutoModelForCausalLM
51
+
52
+ processor = AutoProcessor.from_pretrained("ayoubkirouane/git-base-One-Piece")
53
+ model = AutoModelForCausalLM.from_pretrained("ayoubkirouane/git-base-One-Piece")
54
+ ```