ARMMcBrideT commited on
Commit
936bb8f
·
verified ·
1 Parent(s): 55dce1b

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +12 -0
README.md CHANGED
@@ -10,6 +10,18 @@ pinned: false
10
  <p>Arm’s AI development resources ensure you can deploy at pace, achieving best performance on Arm by default. Our aim is to make your AI development easier, ensuring integration with all major operating systems and AI frameworks, enabling portability for deploying AI on Arm at scale.</p>
11
  <p>Discover below some key resources and content from Arm, including our software libraries and tools, that enable you to optimize for Arm architectures and pass-on significant performance uplift for models – from traditional ML and computer vision workloads to small and large language models - running on Arm-based devices.</p>
12
  <br>
 
 
 
 
 
 
 
 
 
 
 
 
13
  <strong>Arm Kleidi: Unleashing Mass-Market AI Performance on Arm</strong>
14
  <p>Arm Kleidi is a targeted software suite, expediting optimizations for any framework and enabling accelerations for billions of AI workloads across Arm-based devices everywhere. Application developers achieve top performance by default, with no additional work or investment in new skills or tools training required.</p>
15
  <p><b>Useful Resources on Arm Kleidi:</b></p>
 
10
  <p>Arm’s AI development resources ensure you can deploy at pace, achieving best performance on Arm by default. Our aim is to make your AI development easier, ensuring integration with all major operating systems and AI frameworks, enabling portability for deploying AI on Arm at scale.</p>
11
  <p>Discover below some key resources and content from Arm, including our software libraries and tools, that enable you to optimize for Arm architectures and pass-on significant performance uplift for models – from traditional ML and computer vision workloads to small and large language models - running on Arm-based devices.</p>
12
  <br>
13
+ <strong>Meta and Arm: Llama 3.2<br>Accelerated cloud to edge AI performance</strong>
14
+ <p>The availability of smaller LLMs that enable fundamental text-based generative AI workloads, such as Llama 3.2 1B and 3B, are critical to enabling AI inference at scale. Running the new Llama 3.2 3B LLM on Arm-powered mobile devices through the Arm CPU optimized kernel leads to a 5x improvement in prompt processing and 3x improvement in token generation, achieving 19.92 tokens per second in the generation phase. This means less latency when processing AI workloads on the device and a far faster overall user experience. Also, the more AI processed at the edge, the more power that is saved from data traveling to and from the cloud, leading to energy and cost savings.</p>
15
+ <p>Alongside running small models at the edge, we are also able to run larger models, such as Llama 3.2 11B and 90B, in the cloud. The 11B and 90B models are a great fit for CPU based inference workloads in the cloud that generate text and image, as our data on Arm Neoverse V2 shows. When we run the 11B image and text model on the Arm-based AWS Graviton4, we can achieve 29.3 tokens per second in the generation phase. When you consider that the human reading speed is around 5 tokens per second, it’s far outpacing that.</p>
16
+ <p><b>Useful Resources on Arm Kleidi:</b></p>
17
+ <ul><p>
18
+ <li><a href="https://newsroom.arm.com/news/ai-inference-everywhere-with-new-llama-llms-on-arm" target="_blank">Arm Newsroom blog</a></li>
19
+ <li><a href="https://ai.meta.com/blog/llama-3-2-connect-2024-vision-edge-mobile-devices/" target="_blank">Meta Llama 3.2 blog</a></li>
20
+ <li><a href=" https://www.llama.com/docs/getting-the-models/1b3b-partners" target="_blank">Meta Llama 3.2 1b/3/b partner guide</a></li>
21
+ <li><a href="https://www.youtube.com/watch?v=AVqm7SfNQrw" target="_blank">How Arm and Meta are Transforming AI Software Development</a></li>
22
+ <li><a href="https://www.arm.com/markets/artificial-intelligence/software" target="_blank">Arm AI Software Page</a></li>
23
+ </p></ul>
24
+ <br>
25
  <strong>Arm Kleidi: Unleashing Mass-Market AI Performance on Arm</strong>
26
  <p>Arm Kleidi is a targeted software suite, expediting optimizations for any framework and enabling accelerations for billions of AI workloads across Arm-based devices everywhere. Application developers achieve top performance by default, with no additional work or investment in new skills or tools training required.</p>
27
  <p><b>Useful Resources on Arm Kleidi:</b></p>