macadeliccc
/

airoboros-34b-3.2-hqq-8bit

Text Generation

Inference Endpoints

Model card Files Files and versions Community

[jondurbin/airoboros-34b-3.2] [HQQ] [8bit]

This repository contains an HQQ (Half-Quadratic Quantization) model trained on [describe the task or dataset].

Original Repo

Model Details

Model ID: jondurbin/airoboros-34b-3.2
Model Architecture: yi-34b-200k
Training Dataset: Read through original repo
Performance: In progress
nbits: The number of bits used for quantization: 8.
group_size: The size of the quantization group: 128.
quant_zero: Whether to quantize the zero values: True.
quant_scale: Whether to quantize the scale values: True.

Downloads last month: 8

Inference Examples

Text Generation

This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for macadeliccc/airoboros-34b-3.2-hqq-8bit

Base model

jondurbin/airoboros-34b-3.2

Finetuned

(1)

this model