---
language:
- en
base_model: meta-llama/Meta-Llama-3.1-8B-Instruct
pipeline_tag: text-generation
---
# Model Card for Model ID

<!-- Provide a quick summary of what the model is/does. -->

This modelcard aims to be a base template for new models. It has been generated using [this raw template](https://github.com/huggingface/huggingface_hub/blob/main/src/huggingface_hub/templates/modelcard_template.md?plain=1).

## Model Details
meta-llama/Meta-Llama-3.1-8B-Instruct quantized to ONNX GenAI INT4 with Microsoft DirectML optimization

### Model Description
meta-llama/Meta-Llama-3.1-8B-Instruct quantized to ONNX GenAI INT4 with Microsoft DirectML optimization<br>
https://onnxruntime.ai/docs/genai/howto/install.html#directml

Created using ONNX Runtime GenAI's builder.py<br>
https://raw.githubusercontent.com/microsoft/onnxruntime-genai/main/src/python/py/models/builder.py

INT4 accuracy level: FP32 (float32)<br>
8-bit quantization for MoE layers

- **Developed by:** Mochamad Aris Zamroni
- **Model type:** [More Information Needed]
- **Language(s) (NLP):** [More Information Needed]
- **License:** [More Information Needed]
- **Finetuned from model [optional]:** [More Information Needed]

### Model Sources [optional]
https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct

- **Repository:** [More Information Needed]
- **Paper [optional]:** [More Information Needed]
- **Demo [optional]:** [More Information Needed]

## Uses

<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->

### Direct Use
This is Windows DirectML optimized model.

Prerequisites:<br>
1. Install Python 3.10 from Windows Store:<br>
https://apps.microsoft.com/detail/9pjpw5ldxlz5?hl=en-us&gl=US

2. Open command line cmd.exe

3. Create python virtual environment and install onnxruntime-genai-directml<br>
mkdir c:\temp<br>
cd c:\temp<br>
python -m venv dmlgenai<br>
dmlgenai\Scripts\activate.bat<br>
pip install onnxruntime-genai-directml


## How to Get Started with the Model

Use the code below to get started with the model.

[More Information Needed]

#### Preprocessing [optional]

[More Information Needed]


#### Speeds, Sizes, Times [optional]
15 token/s in Radeon 780M with 8GB dedicated RAM

#### Metrics

<!-- These are the evaluation metrics being used, ideally with a description of why. -->

[More Information Needed]

### Results

[More Information Needed]

#### Summary



## Model Examination [optional]

<!-- Relevant interpretability work for the model goes here -->

[More Information Needed]

## Technical Specifications [optional]

### Model Architecture and Objective

[More Information Needed]

### Compute Infrastructure

Microsoft Windows DirectML

#### Hardware

AMD Ryzen 7840U with integrated Radeon 780M GPU
RAM 32GB
shared VRAM 8GB

#### Software

Microsoft Windows DirectML

## Model Card Authors [optional]
Mochamad Aris Zamroni

## Model Card Contact

https://www.linkedin.com/in/zamroni/