Comparison of Quantitative Methods for Set-Based Design When Information Is Uncertain

The systems engineering and product development community can benefit from a methodology that can significantly reduce the likelihood of engineering rework when decisions are made within the context of moderate to high information uncertainty. This condition is predominant in the concept design stage and early lifecycle development stages when data are scarce, model fidelity is low, and stakeholder needs and requirements are expected to change. Several quantitative methods used to develop, reason, and narrow design alternatives have been vulnerable to requirement changes and model fidelity improvements. Methods such as optimization have proven very well when information is certain but have been under scrutiny for situations where it is not. It was hypothesized that classification is more flexible to changes in requirements and information uncertainty than optimization. To test this hypothesis, an observation study was conducted for thirteen scenarios using a limited projectile launcher case study. The scenarios considered changes in performance requirements, material constraints, packaging constraints, and information uncertainty. A novel approach to classification was implemented and evaluated against multi-objective optimization. In all 13 scenarios, classification had more design alternatives satisfying the change in criteria than multi-objective optimization, where eleven of the thirteen scenarios were statically significant (p value less than alpha level of 0.05).