Software architecture evaluation methods based on cost benefit analysis and quantitative decision making

Since many parts of the architecture evaluation steps of the Cost Benefit Analysis Method (CBAM) depend on the stakeholders’ empirical knowledge and intuition, it is very important that such an architecture evaluation method be able to faithfully reflect the knowledge of the experts in determining Architectural Strategy (AS). However, because CBAM requires the stakeholders to make a consensus or vote for collecting data for decision making, it is difficult to accurately reflect the stakeholders’ knowledge in the process. In order to overcome this limitation of CBAM, we propose the two new CBAM-based methods for software architecture evaluation, which respectively adopt the Analytic Hierarchy Process (AHP) and the Analytic Network Process (ANP). Since AHP and ANP use pair-wise comparison they are suitable for a cost and benefit analysis technique since its purpose is not to calculate correct values of benefit and cost but to decide AS with highest return on investment. For that, we first define a generic process of CBAM and develop variations from the generic process by applying AHP and ANP to obtain what we call the CBAM+AHP and CBAM+ANP methods. These new methods not only reflect the knowledge of experts more accurately but also reduce misjudgments. A case study comparison of CBAM and the two new methods is conducted using an industry software project. Because the cost benefit analysis process that we present is generic, new cost benefit analysis techniques with capabilities and characteristics different from the three methods we examine here can be derived by adopting various different constituent techniques.