|
--- |
|
license: mit |
|
--- |
|
|
|
# **Phi-3.5-vision-mlx-int4** |
|
|
|
This is a quantized INT4 model based on Apple MLX Framework Phi-3.5-vision-instruct. You can deploy it on Apple Silicon devices (M1,M2,M3). |
|
|
|
<b><span style="text-decoration:underline">Note: This is unoffical version,just for test and dev.</span></b> |
|
|
|
## **Installation** |
|
|
|
```bash |
|
|
|
pip install -U mlx-vlm |
|
|
|
``` |
|
|
|
## **Conversion** |
|
|
|
```bash |
|
|
|
python -m mlxv_lm.convert --hf-path microsoft/Phi-3.5-vision-instruct -q |
|
|
|
``` |
|
|
|
## **Samples** |
|
|
|
```python |
|
|
|
import mlx.core as mx |
|
from mlx_vlm import load, generate |
|
|
|
model_path = "Your Phi-3.5-vision-mlx-int4 Path" |
|
model, processor = load(model_path,processor_config={"trust_remote_code":"True"}) |
|
|
|
messages = [ |
|
{"role": "user", "content": "<|image|>\n Summarize the pic"}, |
|
] |
|
|
|
prompt = processor.tokenizer.apply_chat_template( |
|
messages, |
|
tokenize=False, |
|
add_generation_prompt=True, |
|
) |
|
|
|
output = generate(model, processor, "Your Image Path", placeholder+prompt, verbose=False, max_tokens=1024) |
|
|
|
output |
|
|
|
|
|
|
|
``` |
