IEEE International Conference on Systems, Man, Cybernetics Presentation: Differentiating ‘Human in the Loop’ Decision Strategies

Recently, research by groups in academia, industry, and government has shifted toward the development of AI and machine learning tools to advise human decision-making in complex, dynamic problems. Within this collaborative environment, humans alone are burdened with the task of managing team strategy due to the AI-agent’s use of an unrealistic model of the human-agent’s decision-making process. This work investigates the use of an unsupervised machine learning method to enable AI-systems to differentiate between human decision-making strategies, enabling improved team collaboration and decision support. An interactive experiment is designed in which human-agents are subjected to a complex decision-making environment (a storm tracking interface) in which the provided visual data sources change over time. Behavioral data from the human-agent is collected, and a k-means clustering algorithm is used to identify individual decision strategies. This approach provides evidence of three distinct decision strategies which demonstrated similar degrees of success as measured by task performance. One cluster utilized a more analytic approach to decision-making, spending more time observing and interacting with each data source, while the other two clusters utilized more heuristic decision-making strategies. These findings indicate that if AI-based decision support systems utilize this approach to distinguish between the human-agent’s decision strategies in real-time, the AI could develop an improved “awareness” of team strategy, enabling better collaboration with human teammates.

Author: swalsh40

Sarah is working toward her PhD in Robotics at Georgia Tech in Atlanta. Her research focuses on the development of shared mental models at the intersection of AI interpretability and human behavior analysis to improve human-AI collaboration in team decision-making tasks. Sarah grew up in Tuckerton, New Jersey. She received her BS in Mathematics from Stockton University and completed her BS in Mechanical Engineering at Rutgers University. After working at the Stevens Institute of Technology and Sandia National Laboratories, Sarah chose to continue her education at Georgia Tech. After graduation, Sarah plans to leverage her training and experience to begin her professional career as a research scientist driving innovation in the fields of artificial intelligence and user-experience.

Leave a Reply

Your email address will not be published. Required fields are marked *