Hugging Face's logo

3-3
/
Venus-120b-v1.0-GGUF

Text Generation
Transformers
English
llama
Not-For-All-Audiences
conversational
Model card Files Community
Venus-120b-v1.0-GGUF / README.md
3-3's picture
3-3
Update README.md
ba542ec
preview code
|
raw
history blame contribute delete
2.45 kB
metadata
base_model: nsfwthrowitaway69/Venus-120b-v1.0
inference: false
language:
  - en
license: llama2
model_creator: nsfwthrowitaway69
model_name: Venus 120B
model_type: llama
pipeline_tag: conversational
quantized_by: 3-3
tags:
  - not-for-all-audiences

Venus 120B v1.0 - GGUF

Description

GGUF quants for nsfwthrowitaway69's Venus 120B v1.0.

Provided quants

Name Quant method Size
Venus-120b-v1.0.Q2_K.gguf Q2_K 50.71 GB
Venus-120b-v1.0.Q3_K_S.gguf Q3_K_S 51.81 GB
Venus-120b-v1.0.Q3_K_M.gguf Q3_K_M 57.64 GB
Venus-120b-v1.0.Q3_K_L.gguf Q3_K_L 63.01 GB
Venus-120b-v1.0.Q4_K_S.gguf Q4_K_S 67.88 GB
Venus-120b-v1.0.Q4_K_M.gguf Q4_K_M 72.14 GB
Venus-120b-v1.0.Q5_K_S.gguf Q5_K_S 82.76 GB
Venus-120b-v1.0.Q5_K_M.gguf Q5_K_M 85.02 GB
Venus-120b-v1.0.Q6_K.gguf Q6_K 98.70 GB
Venus-120b-v1.0.Q8_0.gguf Q8_0 127.84 GB

All of the files are split and require joining

Note: HF does not support uploading files larger than 50GB. Therefore the quants have been uploaded as split files.

Q2_K - Q6_K

Download the two parts of your preferred quant. For Q6_K that would be:

  • Venus-120b-v1.0.Q6_K.gguf-split-a
  • Venus-120b-v1.0.Q6_K.gguf-split-b

Q8_0

Download the three parts of the Q8_0 quant:

  • Venus-120b-v1.0.Q8_0.gguf-split-a
  • Venus-120b-v1.0.Q8_0.gguf-split-b
  • Venus-120b-v1.0.Q8_0.gguf-split-c

To join the files, do the following:

Linux and macOS: 

cat Venus-120b-v1.0.Q6_K.gguf-split-* > Venus-120b-v1.0.Q6_K.gguf && rm Venus-120b-v1.0.Q6_K.gguf-split-*

Note: Replace Q6_K with the quant you downloaded

Windows command line:

Q2_K - Q6_K 

COPY /B Venus-120b-v1.0.Q6_K.gguf-split-a + Venus-120b-v1.0.Q6_K.gguf-split-b Venus-120b-v1.0.Q6_K.gguf
del Venus-120b-v1.0.Q6_K.gguf-split-a Venus-120b-v1.0.Q6_K.gguf-split-b

Note: Replace Q6_K with the quant you downloaded

Q8_K 

COPY /B Venus-120b-v1.0.Q8_0.gguf-split-a + Venus-120b-v1.0.Q8_0.gguf-split-b + Venus-120b-v1.0.Q8_0.gguf-split-c Venus-120b-v1.0.Q8_0.gguf
del Venus-120b-v1.0.Q8_0.gguf-split-a Venus-120b-v1.0.Q8_0.gguf-split-b Venus-120b-v1.0.Q8_0.gguf-split-c