File size: 998 Bytes
4dd5ec2 6c7c7f6 4dd5ec2 6c7c7f6 4dd5ec2 6c7c7f6 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
Model Open Ended VQA: % Human Rating Multiple Choice VQA: % Accuracy Hints-Multiple Choice VQA: % Accuracy Attributions-Multiple Choice VQA: % Accuracy Refernce Based-Automatic Evaluation: Accuracy of Judge Prediction Compared to Human Ratings Refernce Free-Automatic Evaluation: Accuracy of Judge Prediction Compared to Human Ratings Automatic Evaluation: % Auto-Rater Ratings Hints-Automatic Evaluation: % Auto-Rater Ratings Attributions-Automatic Evaluation: % Auto-Rater Ratings
Humans 82 * * * * * 78 * *
Gemini Pro 1.5 40 38 66 72 87 52 53 62 29
Gemini Pro Vision 30 41 62 * 75 38 34 47
GPT4 34 45 69 82 86 51 38 61 25
LlaVA-1.6-34B 15 24 30 * 76 43 21 16 *
LlaVA-1.5-7B 13 17 29 * 70 35 19 30 *
InstructBlip 13 * * * * * 20 28 *
Gemini Pro 1.5 Caption _ Gemini Pro 1.5 23 * * * * * * * *
Human (Oracle) Caption _ Gemini Pro 1.5 50 * * * * * * * *
Claude 3.5 Sonnet * 46 45 * * * 39 * *
GPT4o * 55 83 * * * 50 * *
Qwen-VL-Max * 35 53 * * * 26 * *
Molmo-7B * 34 42 * * * 36 * * |