A new scientific study conducted by Oregon Health & Science University delivers surprising results: ChatGPT and other AI models perform better than the majority of students on biomedical informatics knowledge tests.
The most important facts in a nutshell
- AI models performed better than 75% of students on tests.
- Six different AI models were tested.
- AI performed particularly well on multiple-choice questions and written answers.
- The results raise questions about the assessment of academic performance.
ChatGPT beats humans
The study, which was published on Nature.com, compared the results of 139 students with those of six AI models. The researchers applied the AIs to an online introductory course in biomedical informatics and tested their performance on multiple-choice questions and written exams. The best AI models performed significantly better than three quarters of the students. While not all AI models were equally successful, they were all in the upper range of the results table.
ChatGPT and co. require consequences
According to the authors of the study, this raises crucial questions for the future of academic assessment. In particular, the possibility of cheating and the influence of AIs on learning behavior must be viewed critically. The head of the study, Dr. William Hersh, emphasizes that although continuous updates in scientific fields represent a challenge for AI models, new examination formats are needed that could limit the influence of AIs.
The results of the study suggest that academic institutions need to think about new assessment methods. An adaptation of exams and curricula seems inevitable to control the influence of AI tools and ensure that students continue to acquire real knowledge.
Conclusion
The study shows that AI models such as ChatGPT can outperform human skills in certain areas. This presents educational institutions with the challenge of adapting exams and considering AIs as a potential aid or competitor. A closer look at the use of AIs in an academic context is therefore becoming increasingly important.
