======================================================================== ACCURACY EVALUATION — RESULTS SUMMARY Generated : 2026-05-30 20:36:48 Models : 14 ======================================================================== PER-MODEL RESULTS ------------------------------------------------------------------------ #1 google_gemini-3.1-pro-preview Overall accuracy : 75.7% (227 correct / 300 total | 73 incorrect) Accuracy by modality: CT 83.7% MRI 79.5% X-RAY 65.8% Accuracy by organ category: ABDOMEN 87.3% MSK 81.5% NEURO 72.2% BREAST 71.9% CHEST 64.1% ------------------------------------------------------------------------ #2 google_gemini-2.5-pro-preview Overall accuracy : 65.0% (195 correct / 300 total | 105 incorrect) Accuracy by modality: CT 79.6% MRI 63.6% X-RAY 53.5% Accuracy by organ category: ABDOMEN 73.2% MSK 70.4% BREAST 63.2% CHEST 59.4% NEURO 57.4% ------------------------------------------------------------------------ #3 anthropic_claude-opus-4.7 Overall accuracy : 64.0% (192 correct / 300 total | 108 incorrect) Accuracy by modality: CT 76.5% MRI 58.0% X-RAY 57.9% Accuracy by organ category: ABDOMEN 74.6% NEURO 66.7% CHEST 62.5% BREAST 59.6% MSK 53.7% ------------------------------------------------------------------------ #4 openai_gpt-5.4 Overall accuracy : 62.7% (188 correct / 300 total | 112 incorrect) Accuracy by modality: X-RAY 69.3% CT 63.3% MRI 53.4% Accuracy by organ category: CHEST 68.8% BREAST 66.7% MSK 63.0% ABDOMEN 62.0% NEURO 51.9% ------------------------------------------------------------------------ #5 gemma4_31b_cloud Overall accuracy : 58.3% (175 correct / 300 total | 125 incorrect) Accuracy by modality: CT 68.4% X-RAY 53.5% MRI 53.4% Accuracy by organ category: ABDOMEN 71.8% MSK 59.3% NEURO 57.4% BREAST 54.4% CHEST 46.9% ------------------------------------------------------------------------ #6 openai_gpt-4.1 Overall accuracy : 46.0% (138 correct / 300 total | 162 incorrect) Accuracy by modality: CT 49.0% MRI 45.5% X-RAY 43.9% Accuracy by organ category: MSK 53.7% ABDOMEN 46.5% NEURO 44.4% CHEST 43.8% BREAST 42.1% ------------------------------------------------------------------------ #7 qwen3_vl_235b_cloud Overall accuracy : 42.3% (127 correct / 300 total | 173 incorrect) Accuracy by modality: CT 43.9% X-RAY 42.1% MRI 40.9% Accuracy by organ category: ABDOMEN 52.1% BREAST 45.6% MSK 42.6% NEURO 38.9% CHEST 31.2% ------------------------------------------------------------------------ #8 anthropic_claude-sonnet-4 Overall accuracy : 32.7% (98 correct / 300 total | 202 incorrect) Accuracy by modality: X-RAY 38.6% MRI 31.8% CT 26.5% Accuracy by organ category: MSK 40.7% NEURO 37.0% CHEST 31.2% ABDOMEN 29.6% BREAST 26.3% ------------------------------------------------------------------------ #9 mistral_large_3_cloud Overall accuracy : 24.3% (73 correct / 300 total | 227 incorrect) Accuracy by modality: X-RAY 30.7% MRI 20.5% CT 20.4% Accuracy by organ category: BREAST 35.1% MSK 29.6% ABDOMEN 22.5% NEURO 20.4% CHEST 15.6% ------------------------------------------------------------------------ #10 llama3.2_vision Overall accuracy : 23.3% (70 correct / 300 total | 230 incorrect) Accuracy by modality: X-RAY 27.2% CT 23.5% MRI 18.2% Accuracy by organ category: BREAST 31.6% ABDOMEN 28.2% CHEST 23.4% MSK 18.5% NEURO 13.0% ------------------------------------------------------------------------ #11 medgemma1.5 Overall accuracy : 22.7% (68 correct / 300 total | 232 incorrect) Accuracy by modality: X-RAY 24.6% CT 23.5% MRI 19.3% Accuracy by organ category: MSK 35.2% ABDOMEN 23.9% BREAST 19.3% CHEST 18.8% NEURO 16.7% ------------------------------------------------------------------------ #12 microsoft_llava-med-v1.5-mistral-7b Overall accuracy : 18.3% (55 correct / 300 total | 245 incorrect) Accuracy by modality: X-RAY 21.1% CT 17.3% MRI 15.9% Accuracy by organ category: BREAST 26.3% MSK 22.2% NEURO 16.7% ABDOMEN 14.1% CHEST 14.1% ------------------------------------------------------------------------ #13 med-flamingo_med-flamingo Overall accuracy : 13.3% (40 correct / 300 total | 260 incorrect) Accuracy by modality: X-RAY 15.8% MRI 13.6% CT 10.2% Accuracy by organ category: MSK 20.4% BREAST 17.5% NEURO 11.1% ABDOMEN 9.9% CHEST 9.4% ------------------------------------------------------------------------ #14 hyzhou_MedVersa Overall accuracy : 11.3% (34 correct / 300 total | 266 incorrect) Accuracy by modality: MRI 11.4% X-RAY 11.4% CT 11.2% Accuracy by organ category: BREAST 15.8% NEURO 13.0% ABDOMEN 11.3% MSK 11.1% CHEST 6.2% ------------------------------------------------------------------------ CROSS-MODEL STATISTICS ------------------------------------------------------------------------ Mean accuracy : 40.0% Median accuracy : 37.5% Std deviation : 21.1% Best overall : google_gemini-3.1-pro-preview (75.7%) Worst overall : hyzhou_MedVersa (11.3%) RANKED LEADERBOARD ------------------------------------------------------------------------ Rank Model Accuracy Correct Total ------------------------------------------------------------------------ #1 google_gemini-3.1-pro-preview 75.7% 227 300 #2 google_gemini-2.5-pro-preview 65.0% 195 300 #3 anthropic_claude-opus-4.7 64.0% 192 300 #4 openai_gpt-5.4 62.7% 188 300 #5 gemma4_31b_cloud 58.3% 175 300 #6 openai_gpt-4.1 46.0% 138 300 #7 qwen3_vl_235b_cloud 42.3% 127 300 #8 anthropic_claude-sonnet-4 32.7% 98 300 #9 mistral_large_3_cloud 24.3% 73 300 #10 llama3.2_vision 23.3% 70 300 #11 medgemma1.5 22.7% 68 300 #12 microsoft_llava-med-v1.5-mistral-7b 18.3% 55 300 #13 med-flamingo_med-flamingo 13.3% 40 300 #14 hyzhou_MedVersa 11.3% 34 300 TOP MODELS BY ORGAN CATEGORY ------------------------------------------------------------------------ ABDOMEN 🥇 google_gemini-3.1-p… (87%) 🥈 anthropic_claude-op… (75%) 🥉 google_gemini-2.5-p… (73%) BREAST 🥇 google_gemini-3.1-p… (72%) 🥈 openai_gpt-5.4 (67%) 🥉 google_gemini-2.5-p… (63%) CHEST 🥇 openai_gpt-5.4 (69%) 🥈 google_gemini-3.1-p… (64%) 🥉 anthropic_claude-op… (62%) MSK 🥇 google_gemini-3.1-p… (82%) 🥈 google_gemini-2.5-p… (70%) 🥉 openai_gpt-5.4 (63%) NEURO 🥇 google_gemini-3.1-p… (72%) 🥈 anthropic_claude-op… (67%) 🥉 gemma4_31b_cloud (57%) TOP MODELS BY MODALITY ------------------------------------------------------------------------ CT 🥇 google_gemini-3.1-p… (84%) 🥈 google_gemini-2.5-p… (80%) 🥉 anthropic_claude-op… (76%) MRI 🥇 google_gemini-3.1-p… (80%) 🥈 google_gemini-2.5-p… (64%) 🥉 anthropic_claude-op… (58%) X-RAY 🥇 openai_gpt-5.4 (69%) 🥈 google_gemini-3.1-p… (66%) 🥉 anthropic_claude-op… (58%) ======================================================================== END OF REPORT ========================================================================