Models
Datasets
Spaces
Posts
Docs
Enterprise
Pricing
Log In
Sign Up

Collections

Discover the best community collections!

Collections including paper arxiv:2404.07129

🔍 Daily Picks in Interpretability & Analysis of LMs

Outstanding research in interpretability and evaluation of language models, summarized

Inferring Functionality of Attention Heads from their Parameters

Paper • 2412.11965 • Published 22 days ago • 2
LatentQA: Teaching LLMs to Decode Activations Into Natural Language

Paper • 2412.08686 • Published 27 days ago • 1
Training Large Language Models to Reason in a Continuous Latent Space

Paper • 2412.06769 • Published 29 days ago • 69
Do I Know This Entity? Knowledge Awareness and Hallucinations in Language Models

Paper • 2411.14257 • Published Nov 21, 2024 • 9

Interpretability

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

Paper • 2404.07129 • Published Apr 10, 2024 • 3

INSTRUCTEVAL: Towards Holistic Evaluation of Instruction-Tuned Large Language Models

Paper • 2306.04757 • Published Jun 7, 2023 • 6
Evaluating Instruction-Tuned Large Language Models on Code Comprehension and Generation

Paper • 2308.01240 • Published Aug 2, 2023 • 2
Can Large Language Models Understand Real-World Complex Instructions?

Paper • 2309.09150 • Published Sep 17, 2023 • 2
Evaluating the Instruction-Following Robustness of Large Language Models to Prompt Injection

Paper • 2308.10819 • Published Aug 17, 2023

Papers - ICL - Residual Head Hypothesis

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

Paper • 2404.07129 • Published Apr 10, 2024 • 3

Papers - XAI - Attention - LayerNorm

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

Paper • 2404.07129 • Published Apr 10, 2024 • 3

Papers - ICL - Phase Change Delay - Large Vocabulary Size

Larger vocab is better compression, but may result in longer training ICL phase change delays due to the slower Induction Head Copy Subcircuit (C)

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

Paper • 2404.07129 • Published Apr 10, 2024 • 3

Papers - ICL - Phase Change - Delay - Classes and Labels

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

Paper • 2404.07129 • Published Apr 10, 2024 • 3

Papers - ICL - Induction Head - Copy vs QK Match

See figure 6: Classes vs labels in columns B and C. Subcircuit B delays phase change on number classes vs C delays on number of labels (dramatically)

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

Paper • 2404.07129 • Published Apr 10, 2024 • 3

Papers - Training - ICL - Induction Circuit Evolution

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

Paper • 2404.07129 • Published Apr 10, 2024 • 3

Papers - ICL - Induction Circuit - Data Dependent Learning

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

Paper • 2404.07129 • Published Apr 10, 2024 • 3

Previous
1
2
3
4
Next

Company

TOS Privacy About Jobs

Website

Models Datasets Spaces Pricing Docs