Category: Presentations

Divya passes Thesis Proposal

11/11/2022:

Divya Srivastava, 5th year Mechanical Engineering student, successfully proposed her PhD thesis entitled ‘Transparency and Operator Performance in Human-Autonomy Teams.’

Summary:

Human-autonomy teams aim to leverage the different strengths of humans and autonomous systems respectively to exceed the individual capabilities of each through collaboration. Highly effective human teams develop and utilize a shared mental model (SMM): a synchronized under- standing of the external world and of the tasks, responsibilities, capabilities, and limits of each team member. Recent works assert that the same should apply to human-autonomy teams; however, con- temporary AI commonly consists of “black box” systems, whose internal processes cannot easily be viewed or interpreted. Users can easily develop inaccurate mental models of such systems, impeding SMM development and thus team performance.

This thesis seeks to support the human’s side of Human-AI SMMs in the context of AI-advised Decision Making, a form of teaming in which an AI suggests a solution to a human operator, who is responsible for the final decision. This work focuses on improving shared situation awareness by providing more context to the AI’s internal processing, which should lead the human to a more accurate mental model of the task and the AI, and improved team performance. It will provide a validated implementation of how human mental models of AI can be elicited and measured by researchers and system designers, a quantitative link between factors that influence human mental models and human-autonomy team performance in the context of explainable AI, and finally, it will offer design guidance for increasing non-algorithmic transparency in human-autonomy teams based on empirical results, so that the guidance can be applied to other domains.

CEC PhD Students Win 1st and 2nd Place Doctoral Doctoral Research Awards at ICHMS 2022

The 2022 IEEE International Conference on Human-Machine Systems (ICHMS), held in Orlando, Florida, conducted a Doctoral Research Award Competition (DAC) or doctoral research contributions. Contributions were ranked for both paper submissions and conference presentations by a conference review team. Two CEC Lab members won awards, Sarah Walsh (5th year Robotics PhD Candidate) and Divya Srivastava (5th year Mechanical Engineering PhD Candidate).

1st Place – Sarah Walsh with co-author Karen Feigh

“Consideration of Strategy-specific Adaptive Decision Support”

2nd Place – Divya Srivastava with co-authors J. Mason Lilly and Karen M. Feigh

“The Impact of Improving Shared Situation Awareness on AI-Advised Decision Making”

3rd Place – Jiancheng Nie with co-authors Yusuke Sugahara and Yukio Takeda

“Design of Wearable Robotic Support Limbs for Walking Assistance Based on Configurable Support Polygon”

Each awardee received a commemorative plaque, identifying the conference and contribution. Awardees also received an honorarium from the conference.

ICHMS22Download
IEEE_ICHMS__Proactive_Intelligent_Decision_Support_for_Heuristic_and_Analytic_Decision_MakersDownload
Divya_IEEE_ICHMS_2022_PaperDownload
Divya-IEEE-ICHMS-Nov2022Download

IEEE International Conference on Systems, Man, and Cybernetics: Impact of Missing Information and Strategy on Decision Making (Best Paper Award)

Performance Decision makers frequently encounter environments without perfect information, in which factors such as the distribution of missing information and estimates of missing information significantly impact decision accuracy and speed. This work presents an experiment which modifies an environment with missing information (total information, option imbalance, cue balance) and examines user estimates of the missing information to understand how accuracy and decision speed respond under time pressure. Results indicate that regardless of the way missing information is estimated, certain distributions of missing information reduce decision accuracy. Results from this work also indicate that beyond information distribution and estimation strategy, differences in decision strategy adopted may explain significant differences in decision performance. High performers tend to ignore a greater percentage of information instead of attempting to estimate it, thereby adopting a strategy more heuristic in nature.

IEEE_SMC_2021_SealyPresentationDownload

IEEE International Conference on Systems, Man, Cybernetics Presentation: Differentiating ‘Human in the Loop’ Decision Strategies

Recently, research by groups in academia, industry, and government has shifted toward the development of AI and machine learning tools to advise human decision-making in complex, dynamic problems. Within this collaborative environment, humans alone are burdened with the task of managing team strategy due to the AI-agent’s use of an unrealistic model of the human-agent’s decision-making process. This work investigates the use of an unsupervised machine learning method to enable AI-systems to differentiate between human decision-making strategies, enabling improved team collaboration and decision support. An interactive experiment is designed in which human-agents are subjected to a complex decision-making environment (a storm tracking interface) in which the provided visual data sources change over time. Behavioral data from the human-agent is collected, and a k-means clustering algorithm is used to identify individual decision strategies. This approach provides evidence of three distinct decision strategies which demonstrated similar degrees of success as measured by task performance. One cluster utilized a more analytic approach to decision-making, spending more time observing and interacting with each data source, while the other two clusters utilized more heuristic decision-making strategies. These findings indicate that if AI-based decision support systems utilize this approach to distinguish between the human-agent’s decision strategies in real-time, the AI could develop an improved “awareness” of team strategy, enabling better collaboration with human teammates.

SMC2021_WalshDownload