Xiao Dong and Caroline C. Hayes.
Uncertainty Visualizations: Helping Decision Makers Become More Aware of Uncertainty and Its Implications.
In Journal of Cognitive Engineering and Decision Making, vol. 6, no. 1, pp. 30--56, 2012.

Links:

Abstract:

Uncertainty is inherent in all real work contexts; it creates ambiguities that make decision making difficult. To help decision makers manage ambiguity, the authors developed and evaluated a domain-independent decision support system (DSS), the Uncertainty DSS. It is designed to help decision makers recognize situations in which uncertainty creates ambiguity in their choices and to identify information that can help reduce that ambiguity. It does so by providing an uncertainty visualization, which shows when the range of possible values for two or more alternatives overlap, indicating that one cannot identify the best alternative given the current information. To evaluate the Uncertainty DSS, the authors created a pared-down version, the Certainty DSS, which provides no uncertainty visualizations. They recruited 22 engineering designers and asked them compare alternative designs from real, ongoing design projects using no DSS, the Certainty DSS, and the Uncertainty DSS. The authors found that without the visualizations, participants did not distinguish between ambiguous and unambiguous choices. However, the Uncertainty DSS improved participants' ability to recognize ambiguous decision situations. Additionally, it increased the likelihood that participants would form plans to seek clarifying information. These results suggest that a relatively simple visualization can change the way in which designers think about decision choices.

Bibtex:

@Article{        dong:2012:UVHD,
  author = 	 {Xiao Dong and Caroline C. Hayes},
  title = 	 {Uncertainty Visualizations: Helping Decision Makers Become More Aware of Uncertainty and Its Implications},
  journal = 	 {Journal of Cognitive Engineering and Decision Making},
  year = 	 {2012},
  volume = 	 {6},
  number = 	 {1},
  pages = 	 {30--56},
  month = 	 {March},
}

Images:

References:

Akhavi, F., & Hayes, C. C. (2008). Creating effective decision aids for complex tasks. Journal of Usability Studies, 3, 152-172.
Altman, C. J., & Bursic, K. M. (1988). Verbal protocol analysis as a method to document engineering student design processes. Journal of Engineering Education, 87, 121-132.
54 Journal of Cognitive Engineering and Decision Making / March 2012
Anderson, J. R. (1993). The development of expertise. In B. G. Buchanan & D. C. Wilkins (Eds.), Readings in knowledge acquisition and learning (pp. 61-77). San Francisco, CA: Morgan Kaufman.
Aughenbaugh, J. M., & Paredis, C. J. (2006). The value of using imprecise probabilities in engineering design. Journal of Mechanical Design, 128, 969-979.
Badke-Schaub, P. (2007, June). Decision making processes and leadership in product development departments. Paper presented at the Eighth International NDM Conference, Pacific Grove, CA.
Bellman, R. E., & Zadeh, L. A. (1970). Decision making in a fuzzy environment. Management Sciences, 17, B141-B164.
Bradley, S. R., & Agogino, A. M. (1994). An intelligent real time design methodology for component selection: An approach to managing uncertainty. Journal of Mechanical Design, 116, 980-998.
Cannon-Bowers, J. A., & Bell, H. H. (1997). Training decision makers for complex environ- ments: Implications of the naturalistic decision making perspective. In C. Zsambok & G. Klein (Eds.), Naturalistic decision making (pp. 99-110). Mahwah, NJ: Lawrence Erlbaum.
Chi, M., Glaser, R., & Farr, M. J. (1988). The nature of expertise. Mahwah, NJ: Lawrence Erlbaum.
Cole, W. G. (1989). Understanding Bayesian reasoning via graphical displays. In Proceedings of CHI'89 Human Factors in Computing Systems (pp. 381-386). Austin, TX: ACM Press.
Dwarakanatha, S., & Wallace, K. M. (1995). Decision-making in engineering design: Observa- tions from design experiments. Journal of Engineering Education, 6, 191-206.
Dym, C. L. (1994). Engineering design: A synthesis of views. Cambridge, UK: Cambridge Uni- versity Press.
Erev, I., Bornstein, G., & Wallsten, T. W. (1993). The negative effect of probability assessment on decision quality. Organizational Behavior and Human Decision Processes, 55, 78-94.
Hayes, C. C., & Akhavi, F. (2009, June). Combining naturalistic and mathematical decision aids to support product design. Paper presented at NDM9, the 9th International Confer- ence on Naturalistic Decision Making, London, UK.
Hayes, C. C., & Anderson, R. (2006). Benefits of decision support tools for users with differ- ing levels of domain expertise. In Proceedings of the Human Factors and Ergonomics Society 50th Annual Meeting (Paper No. 503) [CD-ROM]. Santa Monica, CA: Human Factors and Ergonomics Society.
Hayes, J. R. (1981). The complete problem solver. Philadelphia, PA: Franklin Institute Press. Hoffman, J. R., Wilkes, M. S., Day, F. C., Bell, D. S., & Higa, J. K. (2006). The roulette wheel:
An aid to informed decision making. PLoS Medicine, 3, 743-748.
Kahneman, D., & Tversky, A. (1973). On the psychology of prediction. Psychological Review,
80, 237-251.
Kahneman, D., & Tversky, A. (1979). Prospect theory: An analysis of decision under risk.
Econometrica, 47, 263-291.
Klein, A. G. (1993). Decision making in action. Management Sciences, 17, B141-B164. Larkin, J., McDermott, J., Simon, D. P., & Simon, H. A. (1980). Expert and novice perfor-
mance in solving physics problems. Science, 208, 1335-1342.
Larson, A. D., & Hayes, C. C. (2005). An assessment of Weasel: A decision support system
to assist in military planning. In Proceedings of the Human Factors and Ergonomic Society 49th Annual Meeting (pp. 287-291). Santa Monica, CA: Human Factors and Ergonomics Society.
Law, W. S. (1996). Evaluating imprecision in engineering design (Doctoral dissertation). California Institute of Technology, Pasadena.
Uncertainty Visualizations 55
Lewis, K. E., Chen, W., & Schmidt, L. C. (2006). Decision making in engineering design. New York: American Society of Mechanical Engineers.
Lipshitz, R., & Strauss, O. (1997). Coping with uncertainty: A naturalistic decision-making analysis. Organizational Behavior and Human Decision Processes, 69, 149-163.
Lloyd, P., & Scott, P. (1994). Discovering the design problem. Design Studies, 15, 125-140. Newell, A. (1973). Artificial intelligence and the concept of mind. In R. C. Schank & K. M. Colby (Eds.), Computer models of thought and language (pp. 1-60). San Francisco, CA:
Freeman.
Orasanu, J., & Connolly, T. (1993). The reinvention of decision-making. In G. A. Klein, J.
Orasanu, R. Calderwood, & C. Zsambok (Eds.), Decision-making in action: Models and
methods (pp. 3-20). Norwood, NJ: Ablex.
Pahl, G., & Beitz, W. (2005). Engineering design: A systematic approach (2nd ed.). London, UK:
Springer-Verlag.
Saaty, T. L. (1980). The analytical hierarchy process. New York, NY: McGraw-Hill.
Simon, H. A. (1973). The structure of ill-structured problems. Artificial Intelligence, 4, 181-201. Simon, H. A. (1981). The sciences of the artificial (2nd ed.). Cambridge, MA: MIT Press. Sniezek, J. A. (1992). Groups under uncertainty: An examination of confidence in group deci-
sion making. Organizational Behavior and Human Decision Processes, 52, 124-155.
Suwa, M., & Tversky, B. (1997). What do architects and students perceive in their design
sketches? A protocol analysis. Design Studies, 18, 385-403.
Thomas, J. C., & Carroll, J. M. (1984). The psychological study of design. In N. Cross (Ed.),
Developments in design methodology (pp. 226-227). Chichester, UK: Wiley.
Thurston, D. L., & Carnahan, J. V. (1992). Fuzzy rating and utility analysis in preliminary design evaluation of multiple attributes. Transactions of the American Society of Mechanical
Engineers, Journal of Mechanical Design, 114, 646-658.
Triantaphyllou, E. (1997). A sensitivity analysis approach for some deterministic multi-criteria
decision methods. Decision Sciences, 28, 151-194.
Triantaphyllou, E., Kovalerchuk, B., Mann, L., & Knapp, G. M. (1995). Determining the most
important criteria in maintenance decision making. Journal of Quality in Maintenance Engi-
neering, 3, 16-28.
Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and Biases. Science,
New Series, 185, 1124-1131.
Tversky, A., & Shafir, E. (2003). Preference, belief, and similarity: Selected writings. Cambridge,
MA: MIT Press.