How to Train Data-Efficient LLMs
Paper
•
2402.09668
•
Published
•
40
Data Efficient Approaches
-
-
LLM2LLM: Boosting LLMs with Novel Iterative Data Enhancement
Paper • 2403.15042 • Published • 25 -
MAGID: An Automated Pipeline for Generating Synthetic Multi-modal Datasets
Paper • 2403.03194 • Published • 12 -
Orca-Math: Unlocking the potential of SLMs in Grade School Math
Paper • 2402.14830 • Published • 24 -
Synthetic Data (Almost) from Scratch: Generalized Instruction Tuning for Language Models
Paper • 2402.13064 • Published • 47 -
GLoRe: When, Where, and How to Improve LLM Reasoning via Global and Local Refinements
Paper • 2402.10963 • Published • 10 -
In Search of Needles in a 10M Haystack: Recurrent Memory Finds What LLMs Miss
Paper • 2402.10790 • Published • 41 -
BitDelta: Your Fine-Tune May Only Be Worth One Bit
Paper • 2402.10193 • Published • 19 -
Rho-1: Not All Tokens Are What You Need
Paper • 2404.07965 • Published • 88 -
LoRA Learns Less and Forgets Less
Paper • 2405.09673 • Published • 87 -
Show, Don't Tell: Aligning Language Models with Demonstrated Feedback
Paper • 2406.00888 • Published • 30 -
Deep Bayesian Active Learning for Preference Modeling in Large Language Models
Paper • 2406.10023 • Published • 2 -
Unlocking Continual Learning Abilities in Language Models
Paper • 2406.17245 • Published • 28 -
Increasing Model Capacity for Free: A Simple Strategy for Parameter Efficient Fine-tuning
Paper • 2407.01320 • Published