Universal Assisted Generation: Faster Decoding with Any Assistant Model
•
52
Lewis Tunstall PRO
lewtun
AI & ML interests
LLMs, LLMs, LLMs
Recent Activity
liked
a Space
about 2 hours ago
reach-vb/2024-ai-timeline
updated
a collection
about 5 hours ago
Scaling Test-Time Compute with Open Models
updated
a Space
about 6 hours ago
HuggingFaceH4/blogpost-scaling-test-time-compute
Articles
Organizations
Posts
4
Post
1679
This paper (
HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMs (2412.18925)) has a really interesting recipe for inducing o1-like behaviour in Llama models:
* Iteratively sample CoTs from the model, using a mix of different search strategies. This gives you something like Stream of Search via prompting.
* Verify correctness of each CoT using GPT-4o (needed because exact match doesn't work well in medicine where there are lots of aliases)
* Use GPT-4o to reformat the concatenated CoTs into a single stream that includes smooth transitions like "hmm, wait" etc that one sees in o1
* Use the resulting data for SFT & RL
* Use sparse rewards from GPT-4o to guide RL training. They find RL gives an average ~3 point boost across medical benchmarks and SFT on this data already gives a strong improvement.
Applying this strategy to other domains could be quite promising, provided the training data can be formulated with verifiable problems!
* Iteratively sample CoTs from the model, using a mix of different search strategies. This gives you something like Stream of Search via prompting.
* Verify correctness of each CoT using GPT-4o (needed because exact match doesn't work well in medicine where there are lots of aliases)
* Use GPT-4o to reformat the concatenated CoTs into a single stream that includes smooth transitions like "hmm, wait" etc that one sees in o1
* Use the resulting data for SFT & RL
* Use sparse rewards from GPT-4o to guide RL training. They find RL gives an average ~3 point boost across medical benchmarks and SFT on this data already gives a strong improvement.
Applying this strategy to other domains could be quite promising, provided the training data can be formulated with verifiable problems!
Post
6579
We outperform Llama 70B with Llama 3B on hard math by scaling test-time compute 🔥
How? By combining step-wise reward models with tree search algorithms :)
We show that smol models can match or exceed the performance of their much larger siblings when given enough "time to think"
We're open sourcing the full recipe and sharing a detailed blog post.
In our blog post we cover:
📈 Compute-optimal scaling: How we implemented DeepMind's recipe to boost the mathematical capabilities of open models at test-time.
🎄 Diverse Verifier Tree Search (DVTS): An unpublished extension we developed to the verifier-guided tree search technique. This simple yet effective method improves diversity and delivers better performance, particularly at large test-time compute budgets.
🧭 Search and Learn: A lightweight toolkit for implementing search strategies with LLMs and built for speed with vLLM
Here's the links:
- Blog post: HuggingFaceH4/blogpost-scaling-test-time-compute
- Code: https://github.com/huggingface/search-and-learn
Enjoy!
How? By combining step-wise reward models with tree search algorithms :)
We show that smol models can match or exceed the performance of their much larger siblings when given enough "time to think"
We're open sourcing the full recipe and sharing a detailed blog post.
In our blog post we cover:
📈 Compute-optimal scaling: How we implemented DeepMind's recipe to boost the mathematical capabilities of open models at test-time.
🎄 Diverse Verifier Tree Search (DVTS): An unpublished extension we developed to the verifier-guided tree search technique. This simple yet effective method improves diversity and delivers better performance, particularly at large test-time compute budgets.
🧭 Search and Learn: A lightweight toolkit for implementing search strategies with LLMs and built for speed with vLLM
Here's the links:
- Blog post: HuggingFaceH4/blogpost-scaling-test-time-compute
- Code: https://github.com/huggingface/search-and-learn
Enjoy!
Collections
4
spaces
20
models
260
lewtun/Qwen2-0.5B-SFT
Updated
lewtun/Qwen2.5-0.5B-SFT-LoRA
Updated
lewtun/Llama-3.1-8B-SFT-LoRA-packing-no-lm-head
Updated
lewtun/Llama-3.1-8B-SFT-LoRA-no-packing
Updated
lewtun/Llama-3.1-8B-SFT-QLoRA-packing
Updated
lewtun/Llama-3.1-8B-SFT-LoRA-packing-no-saved-modules
Updated
lewtun/Llama-3.1-8B-SFT-LoRA-packing
Updated
lewtun/Llama-3.1-8B-SFT-LoRA-packing-pad-token-eos
Updated
lewtun/Llama-3.1-8B-SFT-QLoRA-packing-pad-token-eos
Updated
lewtun/Llama-3.1-8B-SFT-full-packing
Text Generation
•
Updated
•
16
datasets
58
lewtun/math-bon-debug
Preview
•
Updated
•
92
lewtun/test-fast-parser-l1b-v3
Viewer
•
Updated
•
509
•
59
lewtun/test-fast-parser-l1b-v2
Viewer
•
Updated
•
509
•
56
lewtun/test-fast-parser-l1b
Viewer
•
Updated
•
509
•
54
lewtun/test-fast-parser
Viewer
•
Updated
•
25
•
52
lewtun/bon-prm-serverless-batched
Viewer
•
Updated
•
240
•
48
lewtun/ultrafeedback_gkd
Viewer
•
Updated
•
62.1k
•
43
lewtun/ultrafeedback_binarized
Viewer
•
Updated
•
62.1k
•
50
lewtun/the_tome_50k
Viewer
•
Updated
•
50.1k
•
54
lewtun/asr_dummy
Updated
•
35