JavaBench: A Benchmark of Object-Oriented Code Generation for Evaluating Large Language Models

被引：0
|
作者：
Cao, Jialun [1 ,3 ]
Chen, Zhiyong [2 ]
Wu, Jiarong [1 ,3 ]
Cheung, Shing-Chi [1 ,3 ]
Xu, Chang [2 ]
机构：
[1] The Hong Kong University of Science and Technology, Hong Kong, Hong Kong
[2] Nanjing University, Nanjing, China
[3] Guangzhou HKUST Fok Ying Tung, Research Institute, China
来源：
Proceedings - 2024 39th ACM/IEEE International Conference on Automated Software Engineering, ASE 2024 |
基金：
美国国家科学基金会;
关键词：
Class projects - Codegeneration - [!text type='Java']Java[!/text] codes - Language model - Large language model - Object-oriented code - Objectoriented programming (OOP) - Program synthesis - Project levels - Undergraduate students;
D O I：
暂无
中图分类号：
学科分类号：
摘要：
65
引用
收藏
页码：870 / 882
相关论文
共 50 条
  • [1] BioCoder: a benchmark for bioinformatics code generation with large language models
    Tang, Xiangru
    Qian, Bill
    Gao, Rick
    Chen, Jiakang
    Chen, Xinyun
    Gerstein, Mark B.
    BIOINFORMATICS, 2024, 40 : i266 - i276
  • [2] A Comprehensive Framework for Evaluating API-oriented Code Generation in Large Language Models
    Wu, Yixi
    He, Pengfei
    Wang, Zehao
    Wang, Shaowei
    Tian, Yuan
    Chen, Tse-Hsun
    arXiv,
  • [3] Evaluating documenting techniques on frameworks of object-oriented code
    Faculty of Information Technology, Multimedia University, 63100 Cyberjaya, Malaysia
    ICCAIE - IEEE Conf. Comput. Appl. Ind. Electron., (465-470):
  • [4] Towards the Integration of Large Language Models in an Object-Oriented Programming Course
    Cipriano, Bruno Pereira
    PROCEEDINGS OF THE 2024 CONFERENCE INNOVATION AND TECHNOLOGY IN COMPUTER SCIENCE EDUCATION, VOL 2, ITICSE 2024, 2024, : 832 - 833
  • [5] Framework for evaluating code generation ability of large language models
    Yeo, Sangyeop
    Ma, Yu-Seung
    Kim, Sang Cheol
    Jun, Hyungkook
    Kim, Taeho
    ETRI JOURNAL, 2024, 46 (01) : 106 - 117
  • [6] OBJECT-ORIENTED PROGRAMMING WITHOUT AN OBJECT-ORIENTED LANGUAGE
    BOOCH, G
    SEIDEWITZ, E
    START, M
    FIRESMITH, D
    SIGPLAN NOTICES, 1986, 21 (11): : 508 - 508
  • [7] Transforming UML static models into object-oriented code
    Favre, L
    Martínez, L
    Pereira, C
    37TH INTERNATIONAL CONFERENCE ON TECHNOLOGY OF OBJECT-ORIENTED LANGUAGES AND SYSTEMS, PROCEEDINGS, 2000, : 170 - 181
  • [8] A bilingual benchmark for evaluating large language models
    Alkaoud, Mohamed
    PEERJ COMPUTER SCIENCE, 2024, 10
  • [9] AN OBJECT-ORIENTED APPROACH TO EVALUATING MULTIPLE SPECTRAL MODELS
    MAJORAS, RE
    RICHARDSON, WM
    SEYMOUR, RS
    JOURNAL OF RADIOANALYTICAL AND NUCLEAR CHEMISTRY-ARTICLES, 1995, 193 (02): : 207 - 210
  • [10] Snapshot generation in a constructive object-oriented modeling language
    Ferrari, Mauro
    Fiorentini, Camillo
    Momigliano, Alberto
    Ornaghi, Mario
    LOGIC-BASED PROGRAM SYNTHESIS AND TRANSFORMATION, 2008, 4915 : 169 - +
12345