James K J's picture

1 1 1

James K J

james92

·

s1782662

AI & ML interests

Reinforcement Learning, Computer Vision

Recent Activity

liked a model 4 days ago

BAAI/bge-small-en-v1.5

upvoted a collection 4 months ago

replied to Jaward's post 10 months ago

After giving GPU Programming a hands-on try, I have come to appreciate the level of complexity in AI compute: - Existing/leading frameworks (CUDA, OpenCL, DSLs, even Triton), still fall at the mercy of low-level compute that requires deeper understanding and experience. - Ambiguous optimizations methods that will literally drive you mad 🤯 - Triton is cool but not cool enough (high level abstractions that fall back to low level compute issues as you build more specialized kernels) - As for CUDA, optimization requires considering all major components of the GPU (DRAM, SRAM, ALUs) 🤕 - Models today require stallion written GPU kernels to reduce storage and compute cost. - GPTQ was a big save 👍🏼 @karpathy is right expertise in this area is scarce and the reason is quite obvious - uncertainties: we are still struggling to get peak performance from multi-connected GPUs while maintaining precision and reducing cost. May the Scaling Laws favor us lol.

View all activity

Organizations

None yet

james92's activity

New activity in facebook/bart-large-mnli almost 2 years ago

zero-shot classification pipeline and manual pytorch versions are different results

#3 opened over 2 years ago by

New activity in facebook/bart-large-mnli over 2 years ago

zero-shot classification pipeline and manual pytorch versions are different results

#3 opened over 2 years ago by