Update README.md
Browse files
README.md
CHANGED
@@ -19,6 +19,15 @@ tags:
|
|
19 |
## Model Details
|
20 |
meta-llama/Meta-Llama-3.1-8B-Instruct quantized to ONNX GenAI INT4 with Microsoft DirectML optimization.<br>
|
21 |
Output is reformatted that each sentence starts at new line to improve readability.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
22 |
|
23 |
### Model Description
|
24 |
meta-llama/Meta-Llama-3.1-8B-Instruct quantized to ONNX GenAI INT4 with Microsoft DirectML optimization<br>
|
|
|
19 |
## Model Details
|
20 |
meta-llama/Meta-Llama-3.1-8B-Instruct quantized to ONNX GenAI INT4 with Microsoft DirectML optimization.<br>
|
21 |
Output is reformatted that each sentence starts at new line to improve readability.
|
22 |
+
<pre>
|
23 |
+
...
|
24 |
+
currentdecoded = tokenizer_stream.decode(new_token)
|
25 |
+
if re.findall("^[\x2E\x3A\x3B]$", lastdecoded) and currentdecoded.startswith(" ") and (not currentdecoded.startswith(" *")) :
|
26 |
+
currentdecoded = "\n" + currentdecoded.replace(" ", "", 1)
|
27 |
+
print(currentdecoded, end='', flush=True)
|
28 |
+
lastdecoded = currentdecoded
|
29 |
+
...
|
30 |
+
</pre>
|
31 |
|
32 |
### Model Description
|
33 |
meta-llama/Meta-Llama-3.1-8B-Instruct quantized to ONNX GenAI INT4 with Microsoft DirectML optimization<br>
|