Model Selection Made Easy Using Information Theoretics - An Aircraft Propulsion System Modeling Problem

The United States Department of Defense (DoD) is continually looking for ways to improve test and evaluation techniques to ensure systems meet military requirements prior to acquisition. Recently, the DoD has been pursuing the use of statistical methods to improve test and evaluation. This paper highlights statistical methodologies used by the Air Force Test Center to improve aircraft propulsion system Modeling and Simulation (M&S) efforts. The US Air Force has a long history of using M&S (more than 55 years) during aircraft test and evaluation. In the past, M&S usage was primarily in the aircraft performance and flying qualities areas. Now advancing technology and complex integration are resulting in increased M&S use across broader spectrum of technical disciplines, including propulsion. During propulsion testing, models are used to increase system knowledge in T&E areas which include: Test Planning, Execution, Data Analysis and Evaluation. This paper highlights the 412 Test Wing at Edwards AFB first steps to improve aircraft propulsion system T&E through the implementation of statistically defensible model development techniques. Specifically, this paper will provide an example of typical engineer model development strategies based on past experience, system knowledge, relevant physics and subjective evaluations to determine variables used and structure of the model. This paper will also provide insight into a number of statistics-based approaches including stepwise regression, backwards elimination, the inadequacy of using R-squared and an examination into the effects of mulit-collinearity. However, the focus of this paper is on how Information Theory and Akaike's Information Criteria (AIC) can be easily applied to compare a variety of models and determine the best model available. This paper presents an example of these model development methods applied during a development of a predictive model used for evaluating thrust response of an aircraft engine with a new digital engine control. A case will be made that statistical approaches provide a more mathematically rigorous approach for model selection as compared to traditional approaches based on. engineering judgment.