A Systematic Method for Using Critical Decision Method Interviews to Generate a Garden Path Scenario for Human-in-the-Loop Simulations for Training and Evaluation of New Technologies

Strategies for assessing macrocognitive skill must be tailored to work domains and specific tasks. This paper describes one approach for leveraging the critical decision method, a structured interview technique, to develop an assessment method for new technologies. The approach uses garden path scenarios, where initial cues suggest a different (false prime) diagnosis than later cues and thus requires changing the leading diagnosis over time, to assess sensemaking and re-planning skills in the context of tactical combat casualty care. Critical decision method interviews with emergency medicine physicians revealed critical cues specific to particular injuries and common across different injuries, and also provided cases that were used as the foundation for garden path scenarios. Evaluations using this approach with garden path scenarios enables having an objective measure to compare performance with and without a new technology on a continuous variable of the time until landmark events, such as recognition of a critical cue, committing to a likely diagnosis, or ruling out an inaccurate diagnosis. Additional measures include whether or not particular cues are noticed based upon gaze tracking data and think aloud statements, and exams that assess knowledge of anatomy and treatment priorities. Re-planning measures will focus on comparing performance to an expert model such as for tourniquet application, whether or not tasks on a checklist are conducted in the expected order including for prioritizing where to look for patient assessment, action priorities, and the trainee’s ability to link diagnosis to appropriate treatment.

Managing Uncertainty in Paramedics’ Decision Making

The objective of this article is to improve the understanding of uncertainty in paramedics’ work and the strategies they employ to manage uncertainty, and to provide a resource for training novices. Managing uncertainty is an important part of paramedic decision making and may have a direct impact on patient’s health. Yet, uncertainty has not been sufficiently examined in the naturalistic decision-making paradigm. Therefore, in this study I looked at the uncertainty paramedics have to deal with in nonroutine situations and the strategies they use to manage it. I conducted critical decision method interviews with nine paramedics from paramedic-only ambulance crews. To study uncertainty and coping strategies, I used the RAWFS (reduction, assumption-based reasoning, weighing pros and cons, forestalling, and suppression) heuristic framework. As hypothesized, types of uncertainty were predicted by incident phase, and the coping strategies were predicted by both incident phase and type of uncertainty. The most prevalent type of uncertainty was inadequate understanding of the situation, and the strategy most used to manage uncertainty was reduction. I describe specific examples of cases of uncertainty and the strategies used by paramedics, with the aim of providing materials for training the novices.

How Analysts Think

In this paper we present our observations of how seven criminal intelligence analysts use inference making and storytelling to create explanations. Adapting the Critical Decision Method, we observe they engage in a process of anchoring to gain traction and initiate further inquires; they engage in a laddering process, where they develop explanations to extend or elaborate their ideas that create bridges to new understanding; and they complement their anchoring and laddering activities by associative questioning to discover what other associations exists. These un-intended associations can lead to insights, or generate new patterns recognisable by intuition. We envisage a better understanding of how analyst think and reason might help design software that encourages insight.

How Analysts Think

In this paper we present three cognitive acts analysts use while solving criminal cases: intuition, leap of faith and insight. We used the Emergent Themes Analysis to find out how these cognitive acts help analysts’ inference making and what are their features. We analyzed the interviews with six analysts that were done with the Critical Decision Method. What we discovered is that intuition, leap of faith and insight relate to each other creating an integrated process while solving problems. We propose that a leap of faith occurs between intuition and insight. This is a preliminary study that we plan to develop further. However, current results are interesting enough that we hope the findings may help us design computer systems that will facilitate the process of solving criminal cases by analysts.Criminal intelligence analysis, analytical reasoning, inference making, intuition, leap of faith, insight, critical decision method.