ChatGPT Performs on the Chinese National Medical Licensing Examination

被引：44

作者：

Wang, Xinyi ^{[1
]}

Gong, Zhenye ^{[1
]}

Wang, Guoxin ^{[1
]}

Jia, Jingdan ^{[1
]}

Xu, Ying ^{[1
]}

Zhao, Jialu ^{[1
]}

Fan, Qingye ^{[1
]}

Wu, Shaun ^{[2
]}

Hu, Weiguo ^{[1
]}

Li, Xiaoyang ^{[1
]}

机构：

[1] Shanghai Jiao Tong Univ, Ruijin Hosp, Sch Med, Dept Med Educ, 197 Ruijin Rd 2, Shanghai 200025, Peoples R China

[2] WORK Med Technol Grp LTD, Hangzhou, Peoples R China

来源：

JOURNAL OF MEDICAL SYSTEMS | 2023年 / 47卷 / 01期

关键词：

ChatGPT; Chinese National Medical Licensing Examination; Medical student;

D O I：

10.1007/s10916-023-01961-0

中图分类号：

R19 [保健组织与事业（卫生事业管理）];

学科分类号：

摘要：

ChatGPT, a language model developed by OpenAI, uses a 175 billion parameter Transformer architecture for natural language processing tasks. This study aimed to compare the knowledge and interpretation ability of ChatGPT with those of medical students in China by administering the Chinese National Medical Licensing Examination (NMLE) to both ChatGPT and medical students. We evaluated the performance of ChatGPT in three years' worth of the NMLE, which consists of four units. At the same time, the exam results were compared to those of medical students who had studied for five years at medical colleges. ChatGPT's performance was lower than that of the medical students, and ChatGPT's correct answer rate was related to the year in which the exam questions were released. ChatGPT's knowledge and interpretation ability for the NMLE were not yet comparable to those of medical students in China. It is probable that these abilities will improve through deep learning.

引用

页数：5

共 50 条

[21] Performance of Generative Pretrained Transformer on the National Medical Licensing Examination in Japan
Tanaka, Yudai
Nakata, Takuto
Aiga, Ko
Etani, Takahide
Muramatsu, Ryota
Katagiri, Shun
Kawai, Hiroyuki
Higashino, Fumiya
Enomoto, Masahiro
Noda, Masao
Kometani, Mitsuhiro
Takamura, Masayuki
Yoneda, Takashi
Kakizaki, Hiroaki
Nomura, Akihiro
PLOS DIGITAL HEALTH, 2024, 3 (01):
[22] This too shall pass: the performance of ChatGPT-3.5, ChatGPT-4 and New Bing in an Australian medical licensing examination
Kleinig, Oliver
Gao, Christina
Bacchi, Stephen
MEDICAL JOURNAL OF AUSTRALIA, 2023, 219 (05)
[23] Reporting of the Medical Licensing Examination
Everett, George Douglas
Isea, Luis
JAMA-JOURNAL OF THE AMERICAN MEDICAL ASSOCIATION, 2020, 323 (02): : 185 - 186
[24] EXAMINATION IS THAT ALL THERE IS TO MEDICAL LICENSING
CASTERLI.RL
JOURNAL OF THE AMERICAN MEDICAL ASSOCIATION, 1972, 220 (12): : 1598 - &
[25] The medical licensing examination debate
Archer, Julian
Lynn, Nick
Roberts, Martin
Gale, Tom
de Bere, Sam Regan
Coombes, Lee
REGULATION & GOVERNANCE, 2017, 11 (03) : 315 - 322
[26] Exploring the proficiency of ChatGPT-4: An evaluation of its performance in the Taiwan advanced medical licensing examination
Lin, Shih-Yi
Chan, Pak Ki
Hsu, Wu-Huei
Kao, Chia-Hung
DIGITAL HEALTH, 2024, 10
[27] ChatGPT-4: An assessment of an upgraded artificial intelligence chatbot in the United States Medical Licensing Examination
Mihalache, Andrew
Huang, Ryan S.
Popovic, Marko M.
Muni, Rajeev H.
MEDICAL TEACHER, 2024, 46 (03) : 366 - 372
[28] Performance and exploration of ChatGPT in medical examination, records and education in Chinese: Pave the way for medical AI
Wang, Hongyan
Wu, Weizhen
Dou, Zhi
He, Liangliang
Yang, Liqiang
INTERNATIONAL JOURNAL OF MEDICAL INFORMATICS, 2023, 177
[29] Augmenting Medical Education: An Evaluation of GPT-4 and ChatGPT in Answering Rheumatology Questions from the Spanish Medical Licensing Examination
Madrid Garcia, Alfredo
Rosales, Zulema
Freites, Dalifer
Perez Sancristobal, Ines
Fernandez, Benjamin
Rodriguez Rodriguez, Luis
ARTHRITIS & RHEUMATOLOGY, 2023, 75 : 4095 - 4097
[30] The New Medical Licensing Examination in Portugal
Ribeiro, Joao Carlos
Villanueva, Tiago
ACTA MEDICA PORTUGUESA, 2018, 31 (06): : 293 - 294

← 1 2 3 4 5 →