cicdatopea
commited on
Update README.md
Browse files
README.md
CHANGED
@@ -12,7 +12,7 @@ base_model:
|
|
12 |
|
13 |
This model is an int4 model with group_size 128 and symmetric quantization of [deepseek-ai/DeepSeek-V3](https://huggingface.co/deepseek-ai/DeepSeek-V3) generated by [intel/auto-round](https://github.com/intel/auto-round) algorithm.
|
14 |
|
15 |
-
**Loading the model in Transformers can be quite slow, especially with CUDA devices(30m-1hours). Consider using an alternative serving framework
|
16 |
|
17 |
Please follow the license of the original model.
|
18 |
|
@@ -160,13 +160,7 @@ Generated: DeepSeek Artificial Intelligence Co., Ltd. (referred to as "DeepSeek"
|
|
160 |
Prompt: hello
|
161 |
Generated: Hello! How can I assist you today? 😊
|
162 |
|
163 |
-
|
164 |
"""
|
165 |
-
|
166 |
-
|
167 |
-
|
168 |
-
|
169 |
-
|
170 |
~~~
|
171 |
|
172 |
|
|
|
12 |
|
13 |
This model is an int4 model with group_size 128 and symmetric quantization of [deepseek-ai/DeepSeek-V3](https://huggingface.co/deepseek-ai/DeepSeek-V3) generated by [intel/auto-round](https://github.com/intel/auto-round) algorithm.
|
14 |
|
15 |
+
**Loading the model in Transformers can be quite slow, especially with CUDA devices(30m-1hours). Consider using an alternative serving framework.** However, we have not tested it on other frameworks due to limited cuda resources.
|
16 |
|
17 |
Please follow the license of the original model.
|
18 |
|
|
|
160 |
Prompt: hello
|
161 |
Generated: Hello! How can I assist you today? 😊
|
162 |
|
|
|
163 |
"""
|
|
|
|
|
|
|
|
|
|
|
164 |
~~~
|
165 |
|
166 |
|