FantasiaFoundry
commited on
Update README.md
Browse filesThanks,
@Virt-io
!
README.md
CHANGED
@@ -9,11 +9,11 @@ tags:
|
|
9 |
---
|
10 |
|
11 |
> [!TIP]
|
12 |
-
> **Credits:**
|
13 |
-
>
|
14 |
-
> Made with love by [**@Lewdiculous**](https://huggingface.co/Lewdiculous) with the handy contributions by [**@SolidSnacke**](https://huggingface.co/SolidSnacke). <br>
|
15 |
> If this proves useful for you, feel free to credit and share the repository and authors.
|
16 |
|
|
|
17 |
> [!WARNING]
|
18 |
> **[Important] Llama-3:**
|
19 |
>
|
@@ -22,11 +22,19 @@ tags:
|
|
22 |
>
|
23 |
> Basically, make sure you're in the latest llama.cpp repo commit, then run the new `convert-hf-to-gguf-update.py` script inside the repo (you will need to provide a huggingface-read-token, and you need to have access to the Meta-Llama-3 repositories – [here](https://huggingface.co/meta-llama/Meta-Llama-3-8B) and [here](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct) – to be sure, so fill the access request forms right away to be able to fetch the necessary files, you also might need to refresh the tokens if it stops working after some time), afterwards you need to manually copy the config files from `llama.cpp\models\tokenizers\llama-bpe` into your downloaded **model** folder, replacing the existing ones. <br>
|
24 |
> Try again and the conversion procress should work as expected.
|
|
|
25 |
|
|
|
26 |
> [!WARNING]
|
27 |
> **Experimental:** <br>
|
28 |
> There is a new experimental script added, `gguf-imat-lossless-for-BF16.py`, which performs the conversions directly from a BF16 GGUF to hopefully generate lossless, or as close to that for now, Llama-3 model quantizations avoiding the recent talked about issues on that topic, it is more resource intensive and will generate more writes in the drive as there's a whole additional conversion step that isn't performed in the previous version. This should only be necessary until we have GPU support for BF16 to run directly without conversion.
|
|
|
29 |
|
|
|
|
|
|
|
|
|
|
|
30 |
|
31 |
Pull Requests with your own features and improvements to this script are always welcome.
|
32 |
|
|
|
9 |
---
|
10 |
|
11 |
> [!TIP]
|
12 |
+
> **Credits:** <br>
|
13 |
+
> Made with love by [**@Lewdiculous**](https://huggingface.co/Lewdiculous) with the handy contributions by [**@SolidSnacke**](https://huggingface.co/SolidSnacke) and [**@Virt-io**](https://huggingface.co/Virt-io). <br>
|
|
|
14 |
> If this proves useful for you, feel free to credit and share the repository and authors.
|
15 |
|
16 |
+
<!--
|
17 |
> [!WARNING]
|
18 |
> **[Important] Llama-3:**
|
19 |
>
|
|
|
22 |
>
|
23 |
> Basically, make sure you're in the latest llama.cpp repo commit, then run the new `convert-hf-to-gguf-update.py` script inside the repo (you will need to provide a huggingface-read-token, and you need to have access to the Meta-Llama-3 repositories – [here](https://huggingface.co/meta-llama/Meta-Llama-3-8B) and [here](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct) – to be sure, so fill the access request forms right away to be able to fetch the necessary files, you also might need to refresh the tokens if it stops working after some time), afterwards you need to manually copy the config files from `llama.cpp\models\tokenizers\llama-bpe` into your downloaded **model** folder, replacing the existing ones. <br>
|
24 |
> Try again and the conversion procress should work as expected.
|
25 |
+
-->
|
26 |
|
27 |
+
<!--
|
28 |
> [!WARNING]
|
29 |
> **Experimental:** <br>
|
30 |
> There is a new experimental script added, `gguf-imat-lossless-for-BF16.py`, which performs the conversions directly from a BF16 GGUF to hopefully generate lossless, or as close to that for now, Llama-3 model quantizations avoiding the recent talked about issues on that topic, it is more resource intensive and will generate more writes in the drive as there's a whole additional conversion step that isn't performed in the previous version. This should only be necessary until we have GPU support for BF16 to run directly without conversion.
|
31 |
+
-->
|
32 |
|
33 |
+
> [!NOTE]
|
34 |
+
> **Linux support (experimental):** <br>
|
35 |
+
> There's an experimental script for Linux, `gguf-imat-lossless-for-BF16-linux.py`. <br>
|
36 |
+
> While I personally can't attest for it, it's worth trying and you can report how well it worked, or not, in your case. <br>
|
37 |
+
> Improvements are very welcome!
|
38 |
|
39 |
Pull Requests with your own features and improvements to this script are always welcome.
|
40 |
|