Ananya8154 commited on
Commit
1bad355
1 Parent(s): ae0ebd8

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -39
README.md CHANGED
@@ -44,45 +44,7 @@ The dataset consists of 3k instances of image-text pairs. Given below is sample
44
 
45
  ## How to use?
46
 
47
- You can use either Transformers library for inferencing this model or you can download this hf repo and use this model as instructed below
48
- Using Transformers:
49
- ```python
50
- from PIL import Image
51
- import torch
52
- from transformers import AutoProcessor, LlavaForConditionalGeneration
53
-
54
- model_id = "Ananya8154/Llava-v1.5-7b-FineTuned-Historic-Art-dataset/llava-ftmodel"
55
- model = LlavaForConditionalGeneration.from_pretrained(
56
- model_id,
57
- torch_dtype=torch.float16,
58
- low_cpu_mem_usage=True,
59
- ).to(0)
60
-
61
- processor = AutoProcessor.from_pretrained(model_id)
62
-
63
- conversation = [
64
- {
65
- {
66
- "from": "human",
67
- "value": "What is this image?"
68
- },
69
- {
70
- "from": "gpt",
71
- "value": ""
72
- }
73
- }
74
- ]
75
- prompt = processor.apply_chat_template(conversation, add_generation_prompt=True)
76
-
77
- image_file = "test.jpg" # You can add image file path here # Use only Historical art image # or use the sample image 'test.jpg'
78
- raw_image = Image.open(image_file)
79
- inputs = processor(prompt, raw_image, return_tensors='pt').to(0, torch.float16)
80
-
81
- output = model.generate(**inputs, max_new_tokens=200, do_sample=False)
82
- print(processor.decode(output[0][2:], skip_special_tokens=True))
83
- ```
84
-
85
- ### OR
86
 
87
  The folder 'llava-v1.5-7b-task-lora' contains the lora weights and the folder 'llava-ftmodel' contains the merged model weights and configurations.
88
  - To use the model:
 
44
 
45
  ## How to use?
46
 
47
+ **Note** - Don't use the model with the transformers 'Use this model' on huggingface, alternatively follow the belows step wise approach for inferencing this model.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
48
 
49
  The folder 'llava-v1.5-7b-task-lora' contains the lora weights and the folder 'llava-ftmodel' contains the merged model weights and configurations.
50
  - To use the model: