Update README.md
Browse files
README.md
CHANGED
|
@@ -23,3 +23,65 @@ I wanted to try ORPO and see if it will better align a biased English model to t
|
|
| 23 |
|
| 24 |
## Evaluation and Results
|
| 25 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 23 |
|
| 24 |
## Evaluation and Results
|
| 25 |
|
| 26 |
+
| Community | Llama-3-8B-Instruct | Arabic-ORPO-Llama-3-8B-Instrcut |
|
| 27 |
+
|----------------------------------|---------------------|----------------------------------|
|
| 28 |
+
| **All** | **0.348** | **0.317** |
|
| 29 |
+
| Abstract Algebra | 0.310 | 0.230 |
|
| 30 |
+
| Anatomy | 0.385 | 0.348 |
|
| 31 |
+
| Astronomy | 0.388 | 0.316 |
|
| 32 |
+
| Business Ethics | 0.480 | 0.370 |
|
| 33 |
+
| Clinical Knowledge | 0.396 | 0.385 |
|
| 34 |
+
| College Biology | 0.347 | 0.299 |
|
| 35 |
+
| College Chemistry | 0.180 | 0.250 |
|
| 36 |
+
| College Computer Science | 0.250 | 0.190 |
|
| 37 |
+
| College Mathematics | 0.260 | 0.280 |
|
| 38 |
+
| College Medicine | 0.231 | 0.249 |
|
| 39 |
+
| College Physics | 0.225 | 0.216 |
|
| 40 |
+
| Computer Security | 0.470 | 0.440 |
|
| 41 |
+
| Conceptual Physics | 0.315 | 0.404 |
|
| 42 |
+
| Econometrics | 0.263 | 0.272 |
|
| 43 |
+
| Electrical Engineering | 0.414 | 0.359 |
|
| 44 |
+
| Elementary Mathematics | 0.320 | 0.272 |
|
| 45 |
+
| Formal Logic | 0.270 | 0.214 |
|
| 46 |
+
| Global Facts | 0.320 | 0.320 |
|
| 47 |
+
| High School Biology | 0.332 | 0.335 |
|
| 48 |
+
| High School Chemistry | 0.256 | 0.296 |
|
| 49 |
+
| High School Computer Science | 0.350 | 0.300 |
|
| 50 |
+
| High School European History | 0.224 | 0.242 |
|
| 51 |
+
| High School Geography | 0.323 | 0.364 |
|
| 52 |
+
| High School Government & Politics| 0.352 | 0.285 |
|
| 53 |
+
| High School Macroeconomics | 0.290 | 0.285 |
|
| 54 |
+
| High School Mathematics | 0.237 | 0.278 |
|
| 55 |
+
| High School Microeconomics | 0.231 | 0.273 |
|
| 56 |
+
| High School Physics | 0.252 | 0.225 |
|
| 57 |
+
| High School Psychology | 0.316 | 0.330 |
|
| 58 |
+
| High School Statistics | 0.199 | 0.176 |
|
| 59 |
+
| High School US History | 0.284 | 0.250 |
|
| 60 |
+
| High School World History | 0.312 | 0.274 |
|
| 61 |
+
| Human Aging | 0.369 | 0.430 |
|
| 62 |
+
| Human Sexuality | 0.481 | 0.321 |
|
| 63 |
+
| International Law | 0.603 | 0.405 |
|
| 64 |
+
| Jurisprudence | 0.491 | 0.370 |
|
| 65 |
+
| Logical Fallacies | 0.368 | 0.276 |
|
| 66 |
+
| Machine Learning | 0.214 | 0.312 |
|
| 67 |
+
| Management | 0.350 | 0.379 |
|
| 68 |
+
| Marketing | 0.521 | 0.547 |
|
| 69 |
+
| Medical Genetics | 0.320 | 0.330 |
|
| 70 |
+
| Miscellaneous | 0.446 | 0.443 |
|
| 71 |
+
| Moral Disputes | 0.422 | 0.306 |
|
| 72 |
+
| Moral Scenarios | 0.248 | 0.241 |
|
| 73 |
+
| Nutrition | 0.412 | 0.346 |
|
| 74 |
+
| Philosophy | 0.408 | 0.328 |
|
| 75 |
+
| Prehistory | 0.429 | 0.349 |
|
| 76 |
+
| Professional Accounting | 0.344 | 0.273 |
|
| 77 |
+
| Professional Law | 0.306 | 0.244 |
|
| 78 |
+
| Professional Medicine | 0.228 | 0.206 |
|
| 79 |
+
| Professional Psychology | 0.337 | 0.315 |
|
| 80 |
+
| Public Relations | 0.391 | 0.373 |
|
| 81 |
+
| Security Studies | 0.469 | 0.335 |
|
| 82 |
+
| Sociology | 0.498 | 0.408 |
|
| 83 |
+
| US Foreign Policy | 0.590 | 0.490 |
|
| 84 |
+
| Virology | 0.422 | 0.416 |
|
| 85 |
+
| World Religions | 0.404 | 0.304 |
|
| 86 |
+
| Average (All Communities) | 0.348 | 0.317 |
|
| 87 |
+
|