Can an application help intelligence analysts engage in better reasoning and produce reports that help decision makers make better decisions? A multidisciplinary team of researchers from 黑料不打烊, the University of Arizona, Colorado State University and SRC Inc. aims to answer this question by developing digital tools for improving reasoning and decision making.
Members of the TRACE team, from left: Lu Xiao, Carsten Oesterlund, Kate Kenski, Jennifer Stromer-Galley, James Folkestad, Rosa Mikeal Martey, Brian McKernan and Debi Plochocki. Not pictured: David Kellen and Lael Schooler.
The team, led by School of Information Studies (iSchool) Professor , will develop TRACE (Trackable Reasoning and Analysis for Collaboration and Evaluation), a web-based application aimed at improving reasoning through the use of techniques鈥攕uch as debate and analogical reasoning鈥攁long with crowdsourcing to enhance analysts’ problem-solving abilities and foster creative thinking in order to provide support and guidance where human reasoning falls short.
The 50-month project is supported by a $11.5 million contract from the 聽(Crowdsourcing Evidence, Reasoning, Argumentation, Thinking and Evaluation) Program of the , an arm of the Office for the Director of National Intelligence, which heads the nation’s intelligence services.
“Our goal is to create a reasoning and reporting application that is not only effective, but also appealing to users by making the process intriguing and fun while not interfering with their natural reasoning and writing abilities,鈥� says Stromer-Galley. 鈥淲hat makes this project unique is that we are rigorously testing every aspect of our application using experimental research methods. When this project is done we will have a proven, effective tool for people to use.鈥�
Improving team communication and creating an application that people actually want to use is a key concern of the TRACE team. “Asking people to change the way they work and collaborate is challenging, and with our focus on the process, not just the product of the work, we’ll be able to offer something that engages people in the different dimensions of analysis,鈥� explains , associate professor in the Department of Journalism and Media Communication at Colorado State University.
鈥淚n other words, we don’t just want people to end up with an accurate, effective analysis, we also want them to find new ways of engaging with the work and with each other while they do so,” says Martey.
The TRACE team aims to investigate the role of several techniques for enhancing reasoning while also promoting better communication and discussion among teams. 鈥淐ommunication is key to effective and gratifying decision-making,鈥� says , associate professor of communication and government and public policy at the University of Arizona. 鈥淭his research gives us an unparalleled opportunity to design an online discussion system that harnesses the best group communication strategies, resulting in successful, thoughtful online interactions.鈥�
Using crowdsourcing to improve reasoning and reduce cognitive bias is an innovative idea that will enable the system to overcome common shortcomings in intelligence work by improving the division of labor and reducing both the systematic and random errors individuals may generate while promoting communication and interaction among teams.
To foster richer user experiences, the TRACE application will apply game-based principles of human-computer interaction (HCI) to create an engaging and intuitive solution that promotes efficiency, accuracy and clarity in analysis. The system, which will be created by , a not-for-profit research and development company, will also employ background software processes, such as machine learning, simple decision trees and advanced natural language processing, to help detect reasoning errors and provide support to a variety of tasks. The application will also adopt responsive and just-in-time mechanisms鈥攃alled nudges鈥攖o guide users through reasoning and reporting, as well as provide recommendations.
In each of the project’s three phases, the TRACE team will conduct a series of experiments with different techniques to identify the best approach to improve human reasoning through the use of digital tools. Those include testing and comparing the effects of different analytic techniques and how different levels of interaction affect reasoning and reporting in order to improve user-interface and application design.
The project benefits from a large team of researchers, practitioners and graduate students with unique abilities and areas of expertise, ranging from human-computer interaction, deliberation, crowdsourcing, game and experimental design, interface and software design, cognitive and decision sciences, as well as computational techniques.
The potential benefits of a platform such as TRACE may go beyond the intelligence community and can provide guidance to those interested in improving group communication and teamwork. 鈥淭hrough TRACE, we hope to improve online communication platforms that help groups make more accurate and satisfying decisions when faced with important and sometimes urgent problem-solving tasks,鈥� says Kenski. 鈥淲hile our system is being designed for the intelligence community, we see our product design as one that can potentially improve group communication in many types of settings, including political, public service and business arenas.鈥�
The team brings together scholars from distinct backgrounds and expertise to tackle the challenge. In addition to Stromer-Galley, Martey and Kenski, the team includes: Research Associate Professor ; Associate Professor and Associate Professor from the iSchool; Professor and Assistant Professor in the Department of Psychology in the ; Professor in the School of Education and Professor in the Department of Psychology, both at Colorado University; Brian McKernan, assistant professor at Sage College; and two experts in intelligence analysis, Roc Myers and Sarah Taylor.
