ONR – Interactive Machine Learning


We are interested in machines that can learn new things from people who are not Machine Learning (ML) experts. We propose a research agenda framed around the human factors (HF) and ML research questions of teaching an agent via demonstration and critique. Ultimately, we will develop a training simulation game with several nonplayer characters, all of which can be easily taught new behaviors by an end-user.

With respect to the Science of Autonomy, this proposal is focused on Interactive Intelligence. We seek to understand how an automated system can partner with a human to learn how to act and reason about a new domain. Interactive learning machines that adapt to the needs of a user have long been a goal of AI research. Machine Learning (ML) promises a way to build adaptive systems while avoiding tedious pre-programming (which in sufficiently complex domains is almost impossible); however, we have yet to see many successful applications where machines learn from everyday users. ML techniques are not designed for input from na¨ıve users, remaining by and large a tool built by experts for experts.
Many prior efforts in designing Machine Learning systems with human input, pose the problem as: “what can I get the human to do to help my machine learn better?” Instead, our goal is for systems to learn from everyday people, we reframe the problem as: “how can machines take better advantage of the input that an everyday person is going to be able to provide?” This approach is Interactive Machine Learning (IML), and brings Human Factors to the problem of Machine Learning. IML has two major complementary research goals: (1) to develop interaction protocols for people to teach an ML agent in a way they find natural and intuitive. (2) to design ML algorithms that take better advantage of a human teacher’s guidance; that is, to understand formally how to optimize the information source that is humans, even when those humans have imperfect models of the learning algorithms or suboptimal policies themselves. Our research agenda addresses both of these IML research questions in two complementary types of learning interactions:

  • Learning from Demonstrations (LfD)—A human teacher provides demonstrations of the desired behavior in a given task domain, from which the agent infers a policy of action.
  • Learning from Critique—A human teacher watches critiques the behavior with high-level feedback.

This project is the joint effort with Dr. Andrea Thomaz (PI), Dr. Charles Isbell and Dr. Mark Riedl.

ONR STTR – Designing Contextual Decision Support Systems


Support improved decision making under high stress, uncertain operational conditions through the development of proactive, context-based decision support aids. The objective of this project is to create a scientifically-principled design specification and prototype concepts for a set of decision aids capable of supporting decision making and judgment across multi-faceted mission with dynamic tasking requirements. The result will be a consistent approach to proactive decision support that will facilitate rapid, affordable development for different functions in the combat center, minimize training, insertion into combat systems, and increase end user adoption and utilization.

ONR – Overall Decision Making Process Simulation


Decision makers are consistently asked to make decisions about the course of action required to achieve mission success regardless of the time pressure and the quantity and quality of information available. To be successful, they will adapt their decision strategies to the environment and even use heuristics, simple rules that use little information and can be processed quickly. To support these decision makers, we are designing proactive decision support systems that support adaptive decision making along a range analytic and heuristic strategies.

NASA Authority & Autonomy 2010-2013


NextGen systems are envisioned to be composed of human and automated agents interacting with dynamic flexibility in the allocation of authority and autonomy. The analysis of such concepts of operation requires methods for verifying and validating that the range of roles and responsibilities potentially assignable to the human and automated agents does not lead to unsafe situations. Such analyses must consider the conditions that could impact system safety including the environment, human behavior and operational procedures, methods of collaboration and organization structures, policies and regulations. 

              Agent-based simulation has shown promise toward modeling such complexity but requires a tradeoff between fidelity and the number of simulation runs that can be explored in a reasonable amount of time. Model checking techniques can verify that the modeled system meets safety properties but they require that the component models are of sufficiently limited scope so as to run to completion. By analyzing simulation traces, model checking can also help to ensure that the simulation’s design meets the intended analysis goals. Thus leveraging these types of analysis methods can help to verify operational concepts addressing the allocation of authority and autonomy. To make the analyses using both techniques more efficient, common representations for model components, methods for identifying the appropriate safety properties, and techniques for determining the set of analyses to run are required.

This project is performed in association with Dr. Ellen Bass of Drexel University, Dr. Elsa Gunter of the University of Illinios, and John Rushby of SRI.