CHALLENGES IN USING A DELPHI METHOD TO FORMALIZE CONCEPTUAL UNDERSTANDING IN FUNCTIONAL REASONING

A project to develop a concept inventory for functional reasoning in engineering design using a Delphi method encountered some unanticipated difficulties at the outset. The anonymous elicitation of a list of candidate concepts from an expert panel was expected to be a straightforward first step in a process in which the main work would be the creation and validation of multiple choice questions to probe understanding of these concepts. Low response rates and inconsistent concept descriptions led to a reconsideration of the nature of the concept inventory project. The early results from the project are discussed and compared with a similar stage of a different concept inventory project in thermodynamics. Implications of this comparison for the field of functional reasoning and the development of concept inventories for relatively immature fields are discussed.