ScB: Symptom Checking Benchmarking
Symptom checking benchmarking tests various GenAI models in predicting possible disease causes from only symptoms.
Scoreboard
| Dataset |
Diseases |
OpenAI ChatGPT-4 |
Google Gemini-1.0 |
Baidu Ernie-4 |
Date |
| MCSC Diseases |
181 |
90.5% |
81.38%
|
82.38%
|
20240404 |
| MCSC Symptoms |
194 |
78.71% |
|
|
20230815 |
MCSC: Mayo Clinic Symptom Checker.
References
- Chen A, Chen DO, Tian L. Benchmarking the symptom-checking capabilities of ChatGPT for a broad range of diseases. J Am Med Inform Assoc. 2023:ocad245. [ChatGPT symptom checking benchmarking paper]