Evaluating Test Suite Effectiveness and Assessing Student Code via Constraint Logic Programming

被引：3
|
作者：
Dewey, Kyle [1 ]
Conrad, Phillip [1 ]
Craig, Michelle [2 ]
Morozova, Elena [1 ]
机构：
[1] UC Santa Barbara, Santa Barbara, CA 93106 USA
[2] Univ Toronto, Toronto, ON, Canada
来源：
ITICSE'17: PROCEEDINGS OF THE 2017 ACM CONFERENCE ON INNOVATION AND TECHNOLOGY IN COMPUTER SCIENCE EDUCATION | 2017年
关键词：
D O I：
10.1145/3059009.3059051
中图分类号：
TP301 [理论、方法];
学科分类号：
081202 ;
摘要：
A good suite of test inputs is an indispensable tool both for manual and automated assessment of student submissions to programming assignments. Yet, without a way to evaluate our test suites, it is difficult to know how well we are doing, much less improve our practice. We present a technique for evaluating a hand-generated test suite by comparing its ability to find defects against that of a test suite generated automatically using Constraint Logic Programming (CLP). We describe our technique and present a case study using student submissions for an assignment from a second-year programming course. Our results show that a CLP-generated test suite was able to identify significant defects that the instructor-generated suite missed, despite having similar code coverage.
引用
收藏
页码：317 / 322
页数：6
相关论文
共 24 条
  • [1] Statement frequency coverage: A code coverage criterion for assessing test suite effectiveness
    Aghamohammadi, Alireza
    Mirian-Hosseinabadi, Seyed-Hassan
    Jalali, Sajad
    INFORMATION AND SOFTWARE TECHNOLOGY, 2021, 129 (129)
  • [2] Forming Student Groups with Student Preferences Using Constraint Logic Programming
    Tacadao, Grace
    Toledo, Ramon Prudencio
    ARTIFICIAL INTELLIGENCE: METHODOLOGY, SYSTEMS, AND APPLICATIONS, AIMSA 2016, 2016, 9883 : 259 - 268
  • [3] On the Use of Mutation Analysis for Evaluating Student Test Suite Quality
    Perretta, James
    DeOrio, Andrew
    Guha, Arjun
    Bell, Jonathan
    PROCEEDINGS OF THE 31ST ACM SIGSOFT INTERNATIONAL SYMPOSIUM ON SOFTWARE TESTING AND ANALYSIS, ISSTA 2022, 2022, : 263 - 275
  • [4] Applying Constraint Logic Programming to SQL Test Case Generation
    Caballero, Rafael
    Garcia-Ruiz, Yolanda
    Saenz-Perez, Fernando
    FUNCTIONAL AND LOGIC PROGRAMMING, PROCEEDINGS, 2010, 6009 : 191 - 206
  • [5] Heat exchanger network retrofit via constraint logic programming
    Abbas, HA
    Wiggins, GA
    Lakshmanan, R
    Morton, W
    COMPUTERS & CHEMICAL ENGINEERING, 1999, 23 : S129 - S132
  • [6] Constraint Logic Programming for Real-World Test Laboratory Scheduling
    Geibinger, Tobias
    Mischek, Florian
    Muslin, Nysret
    THIRTY-FIFTH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE, THIRTY-THIRD CONFERENCE ON INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE AND THE ELEVENTH SYMPOSIUM ON EDUCATIONAL ADVANCES IN ARTIFICIAL INTELLIGENCE, 2021, 35 : 6358 - 6366
  • [7] CBUA: A Probabilistic, Predictive, and Practical Approach for Evaluating Test Suite Effectiveness
    Zhang, Peng
    Li, Yanhui
    Ma, Wanwangying
    Yang, Yibiao
    Chen, Lin
    Lu, Hongmin
    Zhou, Yuming
    Xu, Baowen
    IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, 2022, 48 (03) : 1067 - 1096
  • [8] Classifying Test Suite Effectiveness via Model Inference and ROBBDs
    Felbinger, Hermann
    Pill, Ingo
    Wotawa, Franz
    TESTS AND PROOFS, TAP 2016, 2016, 9762 : 76 - 93
  • [9] On factors that impact the relationship between code coverage and test suite effectiveness: a survey
    Barani, Maryam
    Labiche, Yvan
    Rollet, Antoine
    2023 IEEE INTERNATIONAL CONFERENCE ON SOFTWARE TESTING, VERIFICATION AND VALIDATION WORKSHOPS, ICSTW, 2023, : 381 - 388
  • [10] Preamble computation in automated test case generation using constraint logic programming
    Colin, S
    Legeard, B
    Peureux, F
    SOFTWARE TESTING VERIFICATION & RELIABILITY, 2004, 14 (03): : 213 - 235
123