A replication case study to measure the architectural quality of a commercial system

Context: Long-term software management decisions are directly impacted by the quality of the software's architecture. Goal: Herein, we present a replication case study where structural information about a commercial software system is used in conjunction with bug-related change frequencies to measure and predict architecture quality. Method: Metrics describing history and structure were gathered and then correlated with future bug-related issues; the worst of which were visualized and presented to developers. Results: We identified dependencies between components that change together even though they belong to different architectural modules, and as a consequence are more prone to bugs. We validated these dependencies by presenting our results back to the developers. The developers did not identify any of these dependencies as unexpected, but rather architectural necessities. Conclusions: This replication study adds to the knowledge base of CLIO (a tool that detects architectural degradations) by incorporating a new programming language (C++) and by externally replicating a previous case study on a separate commercial code base. Additionally, we provide lessons learned and suggestions for future applications of CLIO.