Probably DPO datasets

davanstrien 's Collections

synthetic-data-generation-demos

sentence-transformers-from-synthetic-data

Synthetic (text) Dataset Generation

haiku

Historic language modeling

Climate

Sourced from Wikimedia

Legal Named Entity Recognition

Top 10% instruction tuning datasets

Top 10 Instruction Tuning Datasets copy

Metadata-generation

MOE papers to read

German Text Embedding Clustering Benchmark datasets

cosmochat-reading-list

datasets-tldr-project

Probably DPO datasets

Probably Alpaca Style Datasets

Direct Preference Optimization Datasets

Image Preference Optimization Datasets

query-to-hub-datasets-viewer-project

updated Jun 26, 2024

A collection of datasets that probably support DPO

Upvote

HuggingFaceH4/ultrafeedback_binarized

Viewer • Updated Oct 16, 2024 • 187k • 6.63k • 260
mlabonne/orpo-dpo-mix-40k

Viewer • Updated Oct 17, 2024 • 44.2k • 929 • 266
argilla/OpenHermesPreferences

Viewer • Updated Mar 1, 2024 • 989k • 665 • 202
argilla/distilabel-capybara-dpo-7k-binarized

Viewer • Updated Jul 16, 2024 • 7.56k • 3.65k • 178
argilla/distilabel-intel-orca-dpo-pairs

Viewer • Updated Feb 5, 2024 • 12.9k • 554 • 169
argilla/dpo-mix-7k

Viewer • Updated Jul 16, 2024 • 7.5k • 520 • 157
argilla/ultrafeedback-binarized-preferences-cleaned

Viewer • Updated Dec 11, 2023 • 60.9k • 6.77k • 127
Dahoas/rm-static

Viewer • Updated Mar 6, 2023 • 81.4k • 5.83k • 113
openbmb/UltraInteract_pair

Viewer • Updated Apr 5, 2024 • 220k • 390 • 106
openbmb/RLAIF-V-Dataset

Viewer • Updated Nov 3, 2024 • 33.8k • 1.32k • 134
llamafactory/DPO-En-Zh-20k

Viewer • Updated Jun 7, 2024 • 20k • 234 • 90
Dahoas/full-hh-rlhf

Viewer • Updated Feb 23, 2023 • 125k • 1.64k • 75
argilla/OpenHermes2.5-dpo-binarized-alpha

Viewer • Updated Feb 10, 2024 • 9.79k • 33 • 64
yitingxie/rlhf-reward-datasets

Viewer • Updated Jan 1, 2023 • 81.4k • 808 • 63
allenai/ultrafeedback_binarized_cleaned

Viewer • Updated Dec 1, 2023 • 186k • 759 • 70
allenai/reward-bench

Viewer • Updated Sep 9, 2024 • 8.11k • 5.38k • 81
CarperAI/openai_summarize_comparisons

Viewer • Updated Feb 27, 2023 • 260k • 1.96k • 40
beyond/rlhf-reward-single-round-trans_chinese

Viewer • Updated Jul 5, 2023 • 24.9k • 63 • 37
argilla/Capybara-Preferences

Viewer • Updated May 9, 2024 • 15.4k • 303 • 40
snorkelai/Snorkel-Mistral-PairRM-DPO-Dataset

Viewer • Updated Jan 23, 2024 • 62.7k • 77 • 42
tasksource/oasst1_pairwise_rlhf_reward

Viewer • Updated Jul 4, 2023 • 18.9k • 89 • 42
wenbopan/Chinese-dpo-pairs

Viewer • Updated Apr 2, 2024 • 10.7k • 84 • 39
maywell/ko_Ultrafeedback_binarized

Viewer • Updated Nov 9, 2023 • 62k • 64 • 29
abacusai/MetaMath_DPO_FewShot

Viewer • Updated Feb 26, 2024 • 395k • 89 • 25
HuggingFaceH4/orca_dpo_pairs

Viewer • Updated Apr 14, 2024 • 12.9k • 115 • 25
Dahoas/static-hh

Viewer • Updated Mar 6, 2023 • 101k • 66 • 20
mlabonne/chatml-OpenHermes2.5-dpo-binarized-alpha

Viewer • Updated Mar 21, 2024 • 9.2k • 56 • 19
HuggingFaceH4/grok-conversation-harmless

Viewer • Updated Feb 2, 2024 • 44.8k • 48 • 23
allenai/preference-test-sets

Viewer • Updated Mar 14, 2024 • 43.2k • 394 • 22
abacusai/ARC_DPO_FewShot

Viewer • Updated Feb 26, 2024 • 4.25k • 63 • 17
cyberagent/chatbot-arena-ja-calm2-7b-chat-experimental

Viewer • Updated Aug 15, 2024 • 29.2k • 110 • 19
vicgalle/configurable-system-prompt-multitask

Viewer • Updated Apr 23, 2024 • 1.95k • 160 • 20
coseal/CodeUltraFeedback_binarized

Viewer • Updated Mar 18, 2024 • 9.5k • 730 • 15
NobodyExistsOnTheInternet/ToxicDPOqa

Viewer • Updated Apr 26, 2024 • 6.87k • 48 • 16
multilingual/orca_dpo_pairs

Viewer • Updated Apr 3, 2024 • 64.7k • 126 • 15
fblgit/simple-math-DPO

Viewer • Updated Aug 1, 2024 • 800k • 87 • 16
HuggingFaceH4/cai-conversation-harmless

Viewer • Updated Feb 2, 2024 • 44.8k • 182 • 15
alvarobartt/dpo-mix-7k-simplified

Viewer • Updated Mar 26, 2024 • 7.5k • 45 • 11
kyujinpy/orca_math_dpo

Viewer • Updated Apr 12, 2024 • 15.3k • 36 • 18
davanstrien/dataset-tldr-preference-dpo

Viewer • Updated Apr 30, 2024 • 522 • 48 • 11
trl-internal-testing/hh-rlhf-trl-style

Viewer • Updated Mar 13, 2024 • 169k • 435 • 11
abacusai/HellaSwag_DPO_FewShot

Viewer • Updated Feb 26, 2024 • 150k • 60 • 8
2A2I/argilla-dpo-mix-7k-arabic

Viewer • Updated Mar 18, 2024 • 7.5k • 48 • 10
argilla/Capybara-Preferences-Filtered

Viewer • Updated Apr 20, 2024 • 14.8k • 63 • 9
2A2I/Aya-Command.R-DPO

Viewer • Updated May 16, 2024 • 14.2k • 30 • 7
kuotient/orca-math-korean-preference

Viewer • Updated Apr 10, 2024 • 193k • 175 • 10
M4-ai/prm_dpo_pairs

Viewer • Updated Jul 1, 2024 • 93.9k • 40 • 7
zerolink/zsql-postgres-dpo

Viewer • Updated Feb 2, 2024 • 259k • 59 • 6
princeton-nlp/llama3-ultrafeedback

Viewer • Updated Jul 18, 2024 • 61.8k • 955 • 16
kaist-ai/Multifaceted-Collection-ORPO

Viewer • Updated Jul 1, 2024 • 64.6k • 42 • 10
kaitchup/UltraFeedback-prompt-chosen-rejected

Viewer • Updated Oct 23, 2023 • 18k • 48 • 6
alexredna/oasst2_dpo_pairs

Viewer • Updated Jul 17, 2024 • 10k • 53 • 6
HuggingFaceH4/grok-conversation-harmless2

Viewer • Updated Jan 19, 2024 • 44.8k • 48 • 7
yongtae-jp/orca_dpo_pairs_ja

Viewer • Updated Jan 9, 2024 • 12.9k • 83 • 6
kaist-ai/Multifaceted-Collection-DPO

Viewer • Updated Jul 1, 2024 • 65.1k • 85 • 8
malhajar/distilabel-intel-orca-dpo-pairs-tr

Viewer • Updated Mar 28, 2024 • 9.12k • 40 • 6
efederici/alpaca-vs-alpaca-orpo-dpo

Viewer • Updated May 15, 2024 • 49.2k • 102 • 7
maywell/ko_Ultrafeedback_binarized_10k

Viewer • Updated Nov 4, 2023 • 10k • 31 • 5
kira/math-dpo

Viewer • Updated Jul 2, 2024 • 2.42k • 38 • 7
mlabonne/distilabel-truthy-dpo-v0.1

Viewer • Updated Feb 12, 2024 • 1.02k • 46 • 5
RLHFlow/UltraFeedback-preference-standard

Viewer • Updated Apr 27, 2024 • 340k • 94 • 9
BramVanroy/orca_dpo_pairs_dutch

Viewer • Updated 29 days ago • 11k • 59 • 5
argilla/ultrafeedback-multi-binarized-preferences-cleaned

Viewer • Updated Dec 11, 2023 • 158k • 115 • 6
malhajar/orca_dpo_pairs-tr

Viewer • Updated Mar 30, 2024 • 10.8k • 55 • 6
lightblue/mitsu_tophalf_borda

Viewer • Updated May 30, 2024 • 1.42k • 37 • 5
NickyNicky/neovalle_H4rmony_dpo_translated_English_to_Spanish

Viewer • Updated May 17, 2024 • 2.02k • 36 • 4
maywell/ko_hh-rlhf-20k_filtered

Viewer • Updated Nov 4, 2023 • 19.4k • 34 • 4
kaist-ai/Multifaceted-Collection-RM

Viewer • Updated Jun 7, 2024 • 64.9k • 46 • 4
HuggingFaceH4/hh-rlhf-h4

Viewer • Updated Nov 29, 2023 • 169k • 98 • 6
orpo-explorers/ultrafeedback_binarized_15k

Viewer • Updated Apr 18, 2024 • 15.4k • 30 • 4
We-Want-GPU/Yi-Ko-DPO-Orca-DPO-Pairs

Viewer • Updated Dec 18, 2023 • 12.7k • 57 • 5
zerolink/zsql-sqlite-dpo

Viewer • Updated Feb 2, 2024 • 260k • 69 • 3
argilla/ultrafeedback-multi-binarized-quality-preferences-cleaned

Viewer • Updated Dec 11, 2023 • 155k • 41 • 4
SAGI-1/Greetings_DPO_dataset_V1

Viewer • Updated Dec 18, 2023 • 891 • 36 • 3
2A2I/Aya-SambaLingo.Arabic.Chat-DPO

Viewer • Updated May 25, 2024 • 14.2k • 40 • 3
AlekseyKorshuk/evol-codealpaca-v1-dpo

Viewer • Updated Jan 25, 2024 • 39.9k • 55 • 4
mlabonne/ultrafeedback-binarized-preferences-cleaned

Viewer • Updated Mar 27, 2024 • 28k • 35 • 3
kuotient/orca-math-korean-dpo-pairs

Viewer • Updated Apr 5, 2024 • 193k • 73 • 9
mii-community/ultrafeedback-preferences-translated-ita

Viewer • Updated Feb 21, 2024 • 60.9k • 39 • 3
sqrti/SPA-VL

Viewer • Updated Jul 3, 2024 • 101k • 408 • 8
HuggingFaceH4/orca_dpo_pairs_no_system_prompt

Viewer • Updated Jan 15, 2024 • 12.9k • 35 • 4
mlabonne/distilabel-intel-orca-dpo-pairs

Viewer • Updated Feb 6, 2024 • 5.92k • 36 • 3
d0rj/rlhf-reward-datasets-ru

Viewer • Updated Jul 2, 2024 • 81.4k • 102 • 4
RLHFlow/HH-RLHF-Harmless-and-RedTeam-standard

Viewer • Updated May 8, 2024 • 42.3k • 42 • 2
2A2I/Aya-AceGPT.13B.Chat-DPO

Viewer • Updated May 16, 2024 • 13k • 44 • 2
RLHFlow/Helpsteer-preference-standard

Viewer • Updated Apr 27, 2024 • 37.1k • 64 • 4
IlyaGusev/saiga_preferences

Viewer • Updated Oct 28, 2024 • 30.6k • 69 • 4
RLHFlow/CodeUltraFeedback-standard

Viewer • Updated Apr 27, 2024 • 50.2k • 45 • 4
RLHFlow/UltraInteract-filtered-standard

Viewer • Updated Apr 28, 2024 • 162k • 46 • 2
RLHFlow/Prometheus2-preference-standard

Viewer • Updated May 5, 2024 • 200k • 47 • 2
RLHFlow/PKU-SafeRLHF-30K-standard

Viewer • Updated Apr 29, 2024 • 26.9k • 75 • 2
RLHFlow/Argilla-Math-DPO-standard

Viewer • Updated Apr 30, 2024 • 2.42k • 45 • 3
droussis/UltraSafety_binarized-orpo-dpo

Viewer • Updated Apr 25, 2024 • 2.26k • 36 • 2
We-Want-GPU/Yi-Ko-DPO-dataset

Viewer • Updated Dec 9, 2023 • 4k • 32 • 4
roborovski/glaive-tool-usage-dpo

Viewer • Updated Feb 29, 2024 • 42k • 32 • 2
vicgalle/OpenHermesPreferences-roleplay

Viewer • Updated Feb 29, 2024 • 3.06k • 44 • 6
GrowingApple/orpo_kor_translated_medical

Viewer • Updated Jun 12, 2024 • 10.4k • 30 • 3
reciprocate/ultrafeedback_cleaned_high_dpo

Viewer • Updated Jan 11, 2024 • 40k • 35 • 2
mlabonne/distilabel-truthy-dpo-v0.1-filtered

Viewer • Updated Feb 15, 2024 • 663 • 33 • 2
xiaodongguaAIGC/awesome-dpo

Viewer • Updated Jul 3, 2024 • 302k • 47 • 3
jan-hq/evol_codealpaca_dpo_binarized

Viewer • Updated Feb 20, 2024 • 39.9k • 36 • 2
2A2I/Aya-Aya.23.8B-DPO

Viewer • Updated May 29, 2024 • 14k • 41 • 2
tasksource/oasst2_pairwise_rlhf_reward

Viewer • Updated Jan 9, 2024 • 28.4k • 44 • 2
NickyNicky/function_call_orpo_sft_phi3_chatML_only

Viewer • Updated May 2, 2024 • 66k • 51 • 2
HuggingFaceH4/summarize_from_feedback

Viewer • Updated Mar 11, 2024 • 353k • 40 • 3
weqweasdas/preference_dataset_mix2

Viewer • Updated Mar 21, 2024 • 528k • 45 • 3
NickyNicky/oasst2_orpo_mix_function_call_phi_3_v1

Viewer • Updated May 2, 2024 • 120k • 32 • 2
BramVanroy/ultra_feedback_dutch

Viewer • Updated 29 days ago • 53.6k • 61 • 2
orpo-explorers/OpenHermesPreferences-10k

Viewer • Updated Mar 30, 2024 • 10k • 31 • 2
orpo-explorers/OpenHermesPreferences-500k

Viewer • Updated Mar 30, 2024 • 500k • 30 • 2
zhengr/ultrafeedback_binarized

Viewer • Updated Nov 8, 2023 • 190k • 96 • 2
ibivibiv/cleaned_orca_math_dpo_pairs

Viewer • Updated May 24, 2024 • 200k • 53 • 2
NickyNicky/nano_finance_200k_en_es_chatML_gemma_orpo_dpo

Viewer • Updated May 29, 2024 • 201k • 40 • 1
Baidicoot/hh-rlhf-golden-harmful

Viewer • Updated May 10, 2024 • 7.64k • 31 • 1
Tachi67/rm_data_motivation

Viewer • Updated Jun 4, 2024 • 44.8k • 30 • 1
trl-internal-testing/hh-rlhf-helpful-base-trl-style

Viewer • Updated May 2, 2024 • 46.2k • 3.22k • 9
flyingfishinwater/ultrafeedback_clean

Viewer • Updated Jul 2, 2024 • 175k • 58 • 1
manishiitg/argilla-distilabel-intel-orca-dpo-pairs

Viewer • Updated Jan 29, 2024 • 23.6k • 49 • 1
Palash123/dpo_anthropic_hh_rlhf

Viewer • Updated Dec 4, 2023 • 168k • 39 • 1
David-Xu/astronomy-stack-dpo-text

Viewer • Updated Mar 8, 2024 • 19.9k • 44 • 2
d0rj/full-hh-rlhf-ru

Viewer • Updated Jul 2, 2024 • 125k • 51 • 2
vwxyzjn/summarize_from_feedback_oai_preprocessing_1706381144

Viewer • Updated Jan 27, 2024 • 179k • 173 • 2
reciprocate/ultrafeedback_orca_math_cleaned_high_dpo

Viewer • Updated Jan 11, 2024 • 48.3k • 30 • 1
HuggingFaceH4/OpenHermes-2.5-preferences-v0-deduped

Viewer • Updated Feb 9, 2024 • 802k • 40 • 1
mlabonne/truthy-dpo-v0.1

Viewer • Updated Feb 18, 2024 • 1.02k • 38 • 1
macadeliccc/distilabel-neurology-preferences-2k-orca-format

Viewer • Updated Feb 22, 2024 • 1.99k • 36 • 1
jan-hq/athirdpath_roleplay_dpo_binarized

Viewer • Updated Jan 9, 2024 • 3.43k • 40 • 1
reciprocate/ultrafeedback_orca_math_dpo

Viewer • Updated Jan 8, 2024 • 73.8k • 33 • 1
d0rj/synthetic-instruct-gptj-pairwise-ru

Viewer • Updated Jul 2, 2024 • 33.1k • 57 • 1
CorticalStack/tak-stack-dpo

Viewer • Updated Feb 28, 2024 • 10.9k • 47 • 1
umarigan/openhermespreference_tr

Viewer • Updated Apr 24, 2024 • 989k • 35 • 3
amztheory/toxic-aira-dataset

Viewer • Updated Apr 30, 2024 • 8.29k • 31 • 1
RLHFlow/HH-RLHF-Helpful-standard

Viewer • Updated Apr 27, 2024 • 115k • 74 • 1
trl-internal-testing/descriptiveness-sentiment-trl-style

Viewer • Updated Apr 9, 2024 • 10.9k • 3.34k • 1
We-Want-GPU/yi-ko-DPO-noprompt-dataset

Viewer • Updated Dec 6, 2023 • 3.83k • 30 • 2
hendrydong/preference_700K

Viewer • Updated Sep 28, 2024 • 700k • 393 • 9
insub/imdb_prefix20_forDPO_gpt2-large-imdb-FT_siebert_sentiment-roberta-large-english

Viewer • Updated Oct 22, 2023 • 50k • 50 • 2
FredrikBL/Legal-Snigel-DPO

Viewer • Updated May 27, 2024 • 2k • 33 • 1
lightblue/mitsu_full_borda

Viewer • Updated May 30, 2024 • 2.85k • 34 • 2
MaziyarPanahi/truthy-dpo-v0.1-axolotl

Viewer • Updated May 2, 2024 • 1.02k • 51 • 7
RLHFlow/Orca-distibalel-standard

Viewer • Updated Apr 28, 2024 • 6.93k • 49 • 1
RLHFlow/prm80k-phase2

Viewer • Updated Apr 28, 2024 • 79.5k • 30 • 4
pythainlp/oasst2_dpo_pairs_enth

Viewer • Updated Jun 16, 2024 • 5.28k • 41 • 1

Upvote

Collection guide
Browse collections