OPEA
/

Safetensors
llama
4-bit precision
intel/auto-round
cicdatopea commited on
Commit
5b5e906
·
verified ·
1 Parent(s): 3f1badb

update readme of must install auto-round from source for asym quantization

Browse files
Files changed (1) hide show
  1. README.md +2 -0
README.md CHANGED
@@ -11,6 +11,8 @@ This model is an int4 model with group_size 128 and asymmetric quantization of [
11
 
12
  HPU: docker image with Gaudi Software Stack is recommended, please refer to following script for environment setup. More details can be found in [Gaudi Guide](https://docs.habana.ai/en/latest/Installation_Guide/Bare_Metal_Fresh_OS.html#launch-docker-image-that-was-built).
13
 
 
 
14
  ```python
15
  from auto_round import AutoHfQuantizer ##must import
16
  import torch
 
11
 
12
  HPU: docker image with Gaudi Software Stack is recommended, please refer to following script for environment setup. More details can be found in [Gaudi Guide](https://docs.habana.ai/en/latest/Installation_Guide/Bare_Metal_Fresh_OS.html#launch-docker-image-that-was-built).
13
 
14
+ CUDA(must install from souce): git clone https://github.com/intel/auto-round && cd auto-round && pip install -vvv --no-build-isolation -e .
15
+
16
  ```python
17
  from auto_round import AutoHfQuantizer ##must import
18
  import torch