Collections

6

AlpaGasus: Training A Better Alpaca with Fewer Data

Paper • 2307.08701 • Published Jul 17, 2023 • 22
The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Dataset

Paper • 2303.03915 • Published Mar 7, 2023 • 6
MADLAD-400: A Multilingual And Document-Level Large Audited Dataset

Paper • 2309.04662 • Published Sep 9, 2023 • 22
SlimPajama-DC: Understanding Data Combinations for LLM Training

Paper • 2309.10818 • Published Sep 19, 2023 • 10

2

AgentInstruct: Toward Generative Teaching with Agentic Flows

Paper • 2407.03502 • Published Jul 3, 2024 • 51
Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing

Paper • 2406.08464 • Published Jun 12, 2024 • 65
Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone

Paper • 2404.14219 • Published Apr 22, 2024 • 253
DataDreamer: A Tool for Synthetic Data Generation and Reproducible LLM Workflows

Paper • 2402.10379 • Published Feb 16, 2024 • 30

AlpaGasus: Training A Better Alpaca with Fewer Data

The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Dataset

MADLAD-400: A Multilingual And Document-Level Large Audited Dataset

SlimPajama-DC: Understanding Data Combinations for LLM Training

AgentInstruct: Toward Generative Teaching with Agentic Flows

Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing

Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone

DataDreamer: A Tool for Synthetic Data Generation and Reproducible LLM Workflows

PDFTriage: Question Answering over Long, Structured Documents

Adapting Large Language Models via Reading Comprehension

Table-GPT: Table-tuned GPT for Diverse Table Tasks

Context-Aware Meta-Learning

Instruction Pre-Training: Language Models are Supervised Multitask Learners

Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality

Perplexed by Perplexity: Perplexity-Based Data Pruning With Small Reference Models

MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark

RLHF Workflow: From Reward Modeling to Online RLHF

Chameleon: Mixed-Modal Early-Fusion Foundation Models

Meteor: Mamba-based Traversal of Rationale for Large Language and Vision Models

An Introduction to Vision-Language Modeling

Perplexed by Perplexity: Perplexity-Based Data Pruning With Small Reference Models

RedPajama: an Open Dataset for Training Large Language Models

Iterative Reasoning Preference Optimization

Better & Faster Large Language Models via Multi-token Prediction

ORPO: Monolithic Preference Optimization without Reference Model

KAN: Kolmogorov-Arnold Networks

Rho-1: Not All Tokens Are What You Need

VASA-1: Lifelike Audio-Driven Talking Faces Generated in Real Time

Instruction-tuned Language Models are Better Knowledge Learners

DoRA: Weight-Decomposed Low-Rank Adaptation

Self-Rewarding Language Models

Self-Discover: Large Language Models Self-Compose Reasoning Structures

OS-Copilot: Towards Generalist Computer Agents with Self-Improvement

Learning From Mistakes Makes LLM Better Reasoner

Watermarking Makes Language Models Radioactive

ChunkAttention: Efficient Self-Attention with Prefix-Aware KV Cache and Two-Phase Partition

GPTVQ: The Blessing of Dimensionality for LLM Quantization

DiLightNet: Fine-grained Lighting Control for Diffusion-based Image Generation