---
library_name: transformers
license: llama3.1
base_model: Replete-AI/L3.1-Pneuma-8B
tags:
- axolotl
- generated_from_trainer
- llama-cpp
- gguf-my-repo
model-index:
- name: L3.1-Pneuma-8B
  results: []
---

# Triangle104/L3.1-Pneuma-8B-Q5_K_M-GGUF
This model was converted to GGUF format from [`Replete-AI/L3.1-Pneuma-8B`](https://huggingface.co/Replete-AI/L3.1-Pneuma-8B) using llama.cpp via the ggml.ai's [GGUF-my-repo](https://huggingface.co/spaces/ggml-org/gguf-my-repo) space.
Refer to the [original model card](https://huggingface.co/Replete-AI/L3.1-Pneuma-8B) for more details on the model.

---
Model details:
-
This model is a fine-tuned version of meta-llama/Llama-3.1-8B-Instruct on the Sandevistan dataset. It achieves the following results on the evaluation set:

Loss: 2.4357

This model is designed to challenge common paradigms in training Large Language Models, giving them a focus on user experience over profitability. These are highly experimental, and need preference training in order to increase their effectiveness. It seems to have retained a large amount of the biases that we were trying to eliminate from the corporate instruct models.

Intended uses & limitations

Chatting, conversation, and assistance in small downstream tasks.

Large Language Models work incredibly differently from humans, so while we are capable of training and rewarding them to act just like us in many ways, you should treat it as a simulation and use the Socratic method when engaging with them. You, as an end-user should always remain in control of your own thoughts and decisions, and use AI as a way to improve yourself rather than becoming dependent on it.

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

    learning_rate: 7.8e-06
    train_batch_size: 8
    eval_batch_size: 8
    seed: 42
    gradient_accumulation_steps: 16
    total_train_batch_size: 128
    optimizer: Use OptimizerNames.PAGED_ADAMW_8BIT with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
    lr_scheduler_type: cosine
    num_epochs: 2

---
## Use with llama.cpp
Install llama.cpp through brew (works on Mac and Linux)

```bash
brew install llama.cpp

```
Invoke the llama.cpp server or the CLI.

### CLI:
```bash
llama-cli --hf-repo Triangle104/L3.1-Pneuma-8B-Q5_K_M-GGUF --hf-file l3.1-pneuma-8b-q5_k_m.gguf -p "The meaning to life and the universe is"
```

### Server:
```bash
llama-server --hf-repo Triangle104/L3.1-Pneuma-8B-Q5_K_M-GGUF --hf-file l3.1-pneuma-8b-q5_k_m.gguf -c 2048
```

Note: You can also use this checkpoint directly through the [usage steps](https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#usage) listed in the Llama.cpp repo as well.

Step 1: Clone llama.cpp from GitHub.
```
git clone https://github.com/ggerganov/llama.cpp
```

Step 2: Move into the llama.cpp folder and build it with `LLAMA_CURL=1` flag along with other hardware-specific flags (for ex: LLAMA_CUDA=1 for Nvidia GPUs on Linux).
```
cd llama.cpp && LLAMA_CURL=1 make
```

Step 3: Run inference through the main binary.
```
./llama-cli --hf-repo Triangle104/L3.1-Pneuma-8B-Q5_K_M-GGUF --hf-file l3.1-pneuma-8b-q5_k_m.gguf -p "The meaning to life and the universe is"
```
or 
```
./llama-server --hf-repo Triangle104/L3.1-Pneuma-8B-Q5_K_M-GGUF --hf-file l3.1-pneuma-8b-q5_k_m.gguf -c 2048
```