Empirical Evidence of the Usage of Programming Languages in the Educational Process

Contribution: A systematic literature review on the empirical evidence regarding the usage of programming languages for learning purposes is presented. The review analyzes different methods and tools at different educational levels and with different objectives. Background: Learning programming has gained relevance in the last decade. This is due to the massive presence of programmable elements ranging from computers to toys. Because of this, the interest of researchers on this topic has increased. Questions, such as what to use, in what educational stages to use it, the effectiveness of the method, and the focal objectives for learning programming are questions that do not have obvious answers. Research Questions: 1) What empirical evidence exists on the use of educational programming languages (EPLs)? 2) In what context is the research performed? 3) How is effectiveness reported in the literature after applying EPLs? 4) What pedagogical goals are achieved by using EPLs? Methodology: Following a formal protocol, automated searches were performed for primary studies from 2007 to 2018. A total of 62 studies were identified, of which 29 were selected and analyzed since they include some type of empirical evidence. Findings: After performing the evaluation, the results support the need for better approaches with empirical evidence when reporting research on the usage of EPLs. Some research opportunities are identified which concerns the used programming languages, the areas or stages of their application, or the need to have more empirical evidence in general and more studies in non-WEIRD (Western, educated, industrialized, rich, and democratic) contexts.