Hugging Face's logo

MaziyarPanahi
/
Ministral-8B-Instruct-2410-GGUF

Text Generation
GGUF
mistral
quantized
2-bit
3-bit
4-bit precision
5-bit
6-bit
8-bit precision
GGUF
conversational
Model card Files Community
1
MaziyarPanahi commited on
Commit
441c9e9
·
verified ·
1 Parent(s): 0fff971

Upload folder using huggingface_hub (#1)

Browse files

- d5a13f617ee3fa535c8de6a9cd1b740ab820efbb9314055c6cf0b19abf610789 (02d9dc8c6dce8b7a77f0a859261b41a6c163daec)
- 301aa5ecad7c89f664ba664a0901ede341189e95449ec01fe3a411b526973d9c (555e076ad2e0ead9b0fc0c2d36e1c9a379f17055)
- 0aad863da0618389e2c3ed5c4c77452e4d848fc3b83acd76eddf9bd748244bda (c12aa8b058301d7b54609443e0e56ad21836afca)
- dc78a82d7020a81249380c744809f5224672ebdcfefbf172ed91507df7d3d5c8 (73281e4b519647b0f9dc64dbbaf1b318df3481b1)
- c8bc163577978a940434061ddb86bb4be32dec6677cdfb8623dfb9da27962c29 (5f9d0bb24576a6efccb144708c213fc5bddd972e)
- b3bd2545b1a74e49426c9412ced4eb8138efdc65fdcd78008afc4e6e0cc1fb28 (59bc7d0695d7a5fe1719b0527558a7ae3ac1f342)
- 806e6792e402c91c14b7cc2a6a32a060f3cbcb71ef591191fdfcb271be147534 (ead37c2f39ed0697d0c514732fface4ef5bddfa6)
- 136f511060487e577ed2e29e794cc2736b3086cd63848c1be504e70d55989b9a (5db985fd8fecc8196e4e220d3c5e0d8aee18a598)
- fcd36bf443816752ce8c17937a2c0d10881b17ac27455ae4006d75864ac7a3db (54465b462fbec77470a85459a5b2fc87d492546f)
- 5f63667595c6597f626e6200a33c952a9909224f6199820c8f78d93e8d7b75d3 (4a43d5f204ade08bd00091c002eade163785b993)
- 085da3157436084c78a548eec6d5a234cc86d6897b80234ec1ef9e1d930de1a6 (cc5f9b0068ddd42bfda3632ce31efafd38244a0f)
- 3f04aa7596332e61603748ace21a531185518bc1adc15118b2f102f015e88193 (01993f699df83a5ee2fe510847f91d44870bc2a7)

Files changed (13) hide show
  1. .gitattributes +11 -0
  2. Ministral-8B-Instruct-2410.Q2_K.gguf +3 -0
  3. Ministral-8B-Instruct-2410.Q3_K_L.gguf +3 -0
  4. Ministral-8B-Instruct-2410.Q3_K_M.gguf +3 -0
  5. Ministral-8B-Instruct-2410.Q3_K_S.gguf +3 -0
  6. Ministral-8B-Instruct-2410.Q4_K_M.gguf +3 -0
  7. Ministral-8B-Instruct-2410.Q4_K_S.gguf +3 -0
  8. Ministral-8B-Instruct-2410.Q5_K_M.gguf +3 -0
  9. Ministral-8B-Instruct-2410.Q5_K_S.gguf +3 -0
  10. Ministral-8B-Instruct-2410.Q6_K.gguf +3 -0
  11. Ministral-8B-Instruct-2410.Q8_0.gguf +3 -0
  12. Ministral-8B-Instruct-2410.fp16.gguf +3 -0
  13. README.md +45 -0
.gitattributes CHANGED
@@ -33,3 +33,14 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
 
 
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ Ministral-8B-Instruct-2410.Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
37
+ Ministral-8B-Instruct-2410.Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
38
+ Ministral-8B-Instruct-2410.Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
39
+ Ministral-8B-Instruct-2410.Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
40
+ Ministral-8B-Instruct-2410.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
41
+ Ministral-8B-Instruct-2410.Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
42
+ Ministral-8B-Instruct-2410.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
43
+ Ministral-8B-Instruct-2410.Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
44
+ Ministral-8B-Instruct-2410.Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
45
+ Ministral-8B-Instruct-2410.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
46
+ Ministral-8B-Instruct-2410.fp16.gguf filter=lfs diff=lfs merge=lfs -text
Ministral-8B-Instruct-2410.Q2_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a5b300793725fbd6173c42b648ffc54633b0858d76be593b3c12bf625738db99
3
+ size 3185478208
Ministral-8B-Instruct-2410.Q3_K_L.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cd9161a19a049e3a619f8087be770a5597ff6683b3bb31bcab8d8996278544f4
3
+ size 4326459968
Ministral-8B-Instruct-2410.Q3_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0c3ddfcae5cebb67a21b8a22fc857d168b2b14ba166d2c0b3a12eca13a73a040
3
+ size 4019227200
Ministral-8B-Instruct-2410.Q3_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2fe5752fe4d1e5bb0a95215b7c522fa14de25e9ceb5698c2432ef8afb07a464d
3
+ size 3664677440
Ministral-8B-Instruct-2410.Q4_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:18bbd947b35bd2a80284ab246cac8189f1e26a5f207afb7711152851150caa05
3
+ size 4911499840
Ministral-8B-Instruct-2410.Q4_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c8e70d0217cacecd38c7949462a413a3327aa4c58c40dd939edb894f1245a087
3
+ size 4685728320
Ministral-8B-Instruct-2410.Q5_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cc6a08aba464b616723daca4e00c942d9a6e41ec1daa1226070aa16d68654c03
3
+ size 5724146240
Ministral-8B-Instruct-2410.Q5_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5cb4c7c483b36b11ae9698cb3a5a6d08ae8f5daa7ed6394175a36b8883567101
3
+ size 5593795136
Ministral-8B-Instruct-2410.Q6_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8097e61c8cf4f16c885c6294c56509079a8505e0be6477c91878ff706aaadb3c
3
+ size 6587583040
Ministral-8B-Instruct-2410.Q8_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:28680bd388070510d69141a0de2944ceb45be163e8279cad99c32450997d3965
3
+ size 8529807936
Ministral-8B-Instruct-2410.fp16.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9387a6ddde33d438678b5c4a5ad213616a38b03f9bee607b7c5b8479817fe182
3
+ size 16048097856
README.md ADDED
@@ -0,0 +1,45 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: mistralai/Ministral-8B-Instruct-2410
3
+ inference: false
4
+ model_creator: mistralai
5
+ model_name: Ministral-8B-Instruct-2410-GGUF
6
+ pipeline_tag: text-generation
7
+ quantized_by: MaziyarPanahi
8
+ tags:
9
+ - quantized
10
+ - 2-bit
11
+ - 3-bit
12
+ - 4-bit
13
+ - 5-bit
14
+ - 6-bit
15
+ - 8-bit
16
+ - GGUF
17
+ - text-generation
18
+ ---
19
+ # [MaziyarPanahi/Ministral-8B-Instruct-2410-GGUF](https://huggingface.co/MaziyarPanahi/Ministral-8B-Instruct-2410-GGUF)
20
+ - Model creator: [mistralai](https://huggingface.co/mistralai)
21
+ - Original model: [mistralai/Ministral-8B-Instruct-2410](https://huggingface.co/mistralai/Ministral-8B-Instruct-2410)
22
+
23
+ ## Description
24
+ [MaziyarPanahi/Ministral-8B-Instruct-2410-GGUF](https://huggingface.co/MaziyarPanahi/Ministral-8B-Instruct-2410-GGUF) contains GGUF format model files for [mistralai/Ministral-8B-Instruct-2410](https://huggingface.co/mistralai/Ministral-8B-Instruct-2410).
25
+
26
+ ### About GGUF
27
+
28
+ GGUF is a new format introduced by the llama.cpp team on August 21st 2023. It is a replacement for GGML, which is no longer supported by llama.cpp.
29
+
30
+ Here is an incomplete list of clients and libraries that are known to support GGUF:
31
+
32
+ * [llama.cpp](https://github.com/ggerganov/llama.cpp). The source project for GGUF. Offers a CLI and a server option.
33
+ * [llama-cpp-python](https://github.com/abetlen/llama-cpp-python), a Python library with GPU accel, LangChain support, and OpenAI-compatible API server.
34
+ * [LM Studio](https://lmstudio.ai/), an easy-to-use and powerful local GUI for Windows and macOS (Silicon), with GPU acceleration. Linux available, in beta as of 27/11/2023.
35
+ * [text-generation-webui](https://github.com/oobabooga/text-generation-webui), the most widely used web UI, with many features and powerful extensions. Supports GPU acceleration.
36
+ * [KoboldCpp](https://github.com/LostRuins/koboldcpp), a fully featured web UI, with GPU accel across all platforms and GPU architectures. Especially good for story telling.
37
+ * [GPT4All](https://gpt4all.io/index.html), a free and open source local running GUI, supporting Windows, Linux and macOS with full GPU accel.
38
+ * [LoLLMS Web UI](https://github.com/ParisNeo/lollms-webui), a great web UI with many interesting and unique features, including a full model library for easy model selection.
39
+ * [Faraday.dev](https://faraday.dev/), an attractive and easy to use character-based chat GUI for Windows and macOS (both Silicon and Intel), with GPU acceleration.
40
+ * [candle](https://github.com/huggingface/candle), a Rust ML framework with a focus on performance, including GPU support, and ease of use.
41
+ * [ctransformers](https://github.com/marella/ctransformers), a Python library with GPU accel, LangChain support, and OpenAI-compatible AI server. Note, as of time of writing (November 27th 2023), ctransformers has not been updated in a long time and does not support many recent models.
42
+
43
+ ## Special thanks
44
+
45
+ 🙏 Special thanks to [Georgi Gerganov](https://github.com/ggerganov) and the whole team working on [llama.cpp](https://github.com/ggerganov/llama.cpp/) for making all of this possible.