MISHANM/Nepali_NLP_eng_to_nepali_Llama3.2_3B_instruction

This model is fine-tuned for the Nepali language, capable of answering queries and translating text from English to Nepali. It leverages advanced natural language processing techniques to provide accurate and context-aware responses.

Model Details

This model is based on meta-llama/Llama-3.2-3B-Instruct and has been LoRA finetuned on Nepali dataset

Training Details

The model is trained on approx 29K instruction samples.

1. GPUs: 2*AMD Instinct MI210
2. Training Time: 4:53:03 Hours
3. train_samples_per_second': 13.153
4. train_steps_per_second': 0.822
5. train_loss': 0.7020601840167722,
6. epoch': 10.0

Inference with HuggingFace

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

# Set the device
device = "cuda" if torch.cuda.is_available() else "cpu"

# Load the fine-tuned model and tokenizer
model_path = "MISHANM/Nepali_NLP_eng_to_nepali_Llama3.2_3B_instruction"
model = AutoModelForCausalLM.from_pretrained(model_path)

# Wrap the model with DataParallel if multiple GPUs are available
if torch.cuda.device_count() > 1:
   print(f"Using {torch.cuda.device_count()} GPUs")
   model = torch.nn.DataParallel(model)

# Move the model to the appropriate device
model.to(device)

tokenizer = AutoTokenizer.from_pretrained(model_path)

# Function to generate text
def generate_text(prompt, max_length=1000, temperature=0.9):
   # Format the prompt according to the chat template
   messages = [
       {
           "role": "system",
           "content": "You are a nepali language expert and linguist, with same knowledge give response in nepali language.",
       },
       {"role": "user", "content": prompt}
   ]

   # Apply the chat template
   formatted_prompt = f"<|system|>{messages[0]['content']}<|user|>{messages[1]['content']}<|assistant|>"

   # Tokenize and generate output
   inputs = tokenizer(formatted_prompt, return_tensors="pt").to(device)
   output = model.module.generate(  # Use model.module for DataParallel
       **inputs, max_new_tokens=max_length, temperature=temperature, do_sample=True
   )
   return tokenizer.decode(output[0], skip_special_tokens=True)

# Example usage
prompt = """Give me a story : The Coming of the King, a legend about King Arthur ."""
translated_text = generate_text(prompt)
print(translated_text)

Citation Information

@misc{MISHANM/Nepali_NLP_eng_to_nepali_Llama3.2_3B_instruction,
  author = {Mishan Maurya},
  title = {Introducing Fine Tuned LLM for Nepali Language},
  year = {2024},
  publisher = {Hugging Face},
  journal = {Hugging Face repository},
  
}