24 1 9

s

Tom-Neverwinter

Tom-Neverwinter

AI & ML interests

Making improvements to help the world.

Recent Activity

reacted to csabakecskemeti's post with 🔥 6 days ago

I've built a small utility to split safetensors file by file. The issue/need came up when I've tried to convert the new Deepseek V3 model from FP8 to BF16. The only Ada architecture GPU I have is an RTX 4080 and the 16GB vram was just wasn't enough for the conversion. BTW: I'll upload the bf16 version here: https://huggingface.co/DevQuasar/deepseek-ai.DeepSeek-V3-Base-bf16 (it will take a while - days with my upload speed) If anyone has access the resources to test it I'd appreciate a feedback if it's working or not. The tool, is available from here: https://github.com/csabakecskemeti/ai_utils/blob/main/safetensor_splitter.py It's splitting every file to n pieces by the layers if possible, and create a new "model.safetensors.index.json" file. I've tested it with Llama 3.1 8B and multiple split sizes, and validated by using inference pipeline. use `--help` for usage Please note current version expects the model is already multiple file and have a "model.safetensors.index.json" layer-safetensor mapping file.

new activity 19 days ago

Apollo-LMMs/README:model pulled

View all activity

Organizations

None yet

Tom-Neverwinter's activity

New activity in Apollo-LMMs/README 19 days ago

model pulled

#1 opened 19 days ago by

Tom-Neverwinter

New activity in multimodalart/flux-lora-the-explorer 5 months ago

how to make a lora

#2 opened 5 months ago by

guardiancc

New activity in meta-llama/Llama-3.1-8B-Instruct 6 months ago

Issues loading model with ooabooga textgenwebui

#20 opened 6 months ago by

Kenji776

New activity in Xenova/whisper-speaker-diarization 6 months ago

how do we run this?

#2 opened 6 months ago by

Tom-Neverwinter

New activity in lmstudio-community/DeepSeek-Coder-V2-Lite-Instruct-GGUF 6 months ago

GGUF for the 236B model

#4 opened 6 months ago by

amarmir

New activity in open-llm-leaderboard/open_llm_leaderboard 6 months ago

WizardLM-8x22B Evaluation failed

#823 opened 6 months ago by

llama-anon

New activity in DavidAU/L3-Stheno-Maid-Blackroot-Grand-HORROR-16B-GGUF 6 months ago

Evil.

#1 opened 6 months ago by

Reithan

New activity in TheDrummer/Llama-3SOME-8B-v2-GGUF 7 months ago

can we have some more details?

#1 opened 7 months ago by

Tom-Neverwinter

New activity in LoneStriker/DeepSeek-Coder-V2-Instruct-GGUF 7 months ago

How good is the gguf?

#3 opened 7 months ago by

Tom-Neverwinter

New activity in ddh0/Phi-3-mini-4k-instruct-bf16-GGUF 8 months ago

censored

#1 opened 8 months ago by

Tom-Neverwinter

New activity in LiteLLMs/Mixtral-8x22B-v0.1-GGUF 9 months ago

ram usage

#1 opened 9 months ago by

Tom-Neverwinter

New activity in bartowski/Mixtral-8x22B-v0.1-GGUF 9 months ago

resources to run

#3 opened 9 months ago by

Tom-Neverwinter

New activity in bartowski/Beyonder-4x7B-v3-exl2 9 months ago

3.0 bpw?

#1 opened 10 months ago by

CulturedMan

New activity in ai21labs/Jamba-v0.1 9 months ago

multiple gpu?

#3 opened 9 months ago by

bdambrosio

New activity in NeverSleep/Noromaid-v0.4-Mixtral-Instruct-8x7b-Zloss-GGUF 9 months ago

Missing config.json

#2 opened 12 months ago by

Cayleb

New activity in vikhyatk/moondream2 9 months ago

enter it into the leaderboard?

#6 opened 9 months ago by

Tom-Neverwinter

New activity in hpcai-tech/Open-Sora 10 months ago

safetensors?

#5 opened 10 months ago by

Tom-Neverwinter

New activity in google/gemma-7b 10 months ago

Dont download, google scuttled this model

#77 opened 10 months ago by

Tom-Neverwinter

New activity in NousResearch/Nous-Hermes-2-Yi-34B-GGUF 10 months ago

is this filtered(cencored) or not? can i run it on 3090?

#1 opened 10 months ago by

Avos0001

New activity in ShinojiResearch/Senku-70B-Full 10 months ago

Hardware

#5 opened 10 months ago by

Tom-Neverwinter