ChatGPT Performs on the Chinese National Medical Licensing Examination

被引：44
|
作者：
Wang, Xinyi [1 ]
Gong, Zhenye [1 ]
Wang, Guoxin [1 ]
Jia, Jingdan [1 ]
Xu, Ying [1 ]
Zhao, Jialu [1 ]
Fan, Qingye [1 ]
Wu, Shaun [2 ]
Hu, Weiguo [1 ]
Li, Xiaoyang [1 ]
机构：
[1] Shanghai Jiao Tong Univ, Ruijin Hosp, Sch Med, Dept Med Educ, 197 Ruijin Rd 2, Shanghai 200025, Peoples R China
[2] WORK Med Technol Grp LTD, Hangzhou, Peoples R China
来源：
JOURNAL OF MEDICAL SYSTEMS | 2023年 / 47卷 / 01期
关键词：
ChatGPT; Chinese National Medical Licensing Examination; Medical student;
D O I：
10.1007/s10916-023-01961-0
中图分类号：
R19 [保健组织与事业（卫生事业管理）];
学科分类号：
摘要：
ChatGPT, a language model developed by OpenAI, uses a 175 billion parameter Transformer architecture for natural language processing tasks. This study aimed to compare the knowledge and interpretation ability of ChatGPT with those of medical students in China by administering the Chinese National Medical Licensing Examination (NMLE) to both ChatGPT and medical students. We evaluated the performance of ChatGPT in three years' worth of the NMLE, which consists of four units. At the same time, the exam results were compared to those of medical students who had studied for five years at medical colleges. ChatGPT's performance was lower than that of the medical students, and ChatGPT's correct answer rate was related to the year in which the exam questions were released. ChatGPT's knowledge and interpretation ability for the NMLE were not yet comparable to those of medical students in China. It is probable that these abilities will improve through deep learning.
引用
收藏
页数：5
相关论文
共 50 条
  • [1] ChatGPT Performs on the Chinese National Medical Licensing Examination
    Xinyi Wang
    Zhenye Gong
    Guoxin Wang
    Jingdan Jia
    Ying Xu
    Jialu Zhao
    Qingye Fan
    Shaun Wu
    Weiguo Hu
    Xiaoyang Li
    Journal of Medical Systems, 47
  • [2] Can ChatGPT pass China's national medical licensing examination?
    Shang, Luxiang
    Xue, Mingyue
    Hou, Yinglong
    Tang, Baopeng
    ASIAN JOURNAL OF SURGERY, 2023, 46 (12) : 6112 - 6113
  • [3] Accuracy of ChatGPT on Medical Questions in the National Medical Licensing Examination in Japan: Evaluation Study
    Yanagita, Yasutaka
    Yokokawa, Daiki
    Uchida, Shun
    Tawara, Junsuke
    Ikusaka, Masatomi
    JMIR FORMATIVE RESEARCH, 2023, 7
  • [4] Artificial intelligence in global health equity: an evaluation and discussion on the application of ChatGPT, in the Chinese National Medical Licensing Examination
    Tong, Wenting
    Guan, Yongfu
    Chen, Jinping
    Huang, Xixuan
    Zhong, Yuting
    Zhang, Changrong
    Zhang, Hui
    FRONTIERS IN MEDICINE, 2023, 10
  • [5] Performance of ChatGPT on Chinese national medical licensing examinations: a five-year examination evaluation study for physicians, pharmacists and nurses
    Zong, Hui
    Li, Jiakun
    Wu, Erman
    Wu, Rongrong
    Lu, Junyu
    Shen, Bairong
    BMC MEDICAL EDUCATION, 2024, 24 (01)
  • [6] Performance of ChatGPT on Chinese national medical licensing examinations: a five-year examination evaluation study for physicians, pharmacists and nurses
    Hui Zong
    Jiakun Li
    Erman Wu
    Rongrong Wu
    Junyu Lu
    Bairong Shen
    BMC Medical Education, 24
  • [7] Performance of ChatGPT on the National Korean Occupational Therapy Licensing Examination
    Lee, Si-An
    Heo, Seoyoon
    Park, Jin-Hyuck
    DIGITAL HEALTH, 2024, 10
  • [8] Original Paper Performance of ChatGPT on the Peruvian National Licensing Medical Examination: Cross-Sectional Study
    Flores-Cohaila, Javier A.
    Garcia-Vicente, Abigail
    Vizcarra-Jimenez, Sonia F.
    De la Cruz-Galan, Janith
    Gutierrez-Arratia, Jesus
    Torres, Blanca Geraldine Quiroga
    Taype-Rondan, Alvaro
    JMIR MEDICAL EDUCATION, 2023, 9
  • [9] The performance evaluation of artificial intelligence ERNIE bot in Chinese National Medical Licensing Examination
    Huang, Leiyun
    Hu, Jinghan
    Cai, Qingjin
    Fu, Guangjie
    Bai, Zhenglin
    Liu, Yongzhen
    Zheng, Ji
    Meng, Zengdong
    POSTGRADUATE MEDICAL JOURNAL, 2024, 100 (1190) : 952 - 953
  • [10] Performance of ChatGPT, Bard, Claude, and Bing on the Peruvian National Licensing Medical Examination: a cross-sectional study
    Torres-Zegarra, Betzy Clariza
    Rios-Garcia, Wagner
    Nana-Cordova, Alvaro Micael
    Arteaga-Cisneros, Karen Fatima
    Chalco, Xiomara Cristina Benavente
    Ordonez, Marina Atena Bustamante
    Rios, Carlos Jesus Gutierrez
    Godoy, Carlos Alberto Ramos
    Quezada, Kristell Luisa Teresa Panta
    Gutierrez-Arratia, Jesus Daniel
    Flores-Cohaila, Javier Alejandro
    JOURNAL OF EDUCATIONAL EVALUATION FOR HEALTH PROFESSIONS, 2023, 20
12345