About me
I’m a PhD student in the School of Interactive Computing. I specialize in Reinforcement- and Imitation Learning or just Machine Learning in general. I joined the program in 2014 and am working with my PhD advisor, Dr. Charles Isbell. Before joining, I obtained my BSc in Computer Science at TU Darmstadt in Germany.
About my research
As a researcher in AI, my goal is to build agents that are able to learn to act intelligently in arbitrary and unknown environments. Imitation learning is one approach to tackle this by teaching agents and robots how to act from expert demonstrations. These demonstrations prove to be a rich learning signal; beyond showing complete solutions, they can teach the agent how to act in situations that look quite dissimilar from situations shown as part of the demonstrations. How to use demonstrations most effectively is a question that has occupied many researchers in the past. In my thesis research, I developed a novel view on this question. Beyond introducing algorithms that utilize this view to imitate more effectively on a number of interesting domains ( including robots), it is my aim to add a novel perspective to facilitate long-term developments in the field of imitation learning.
Generative Predecessor Models for Sample-Efficient Imitation Learning
As I learned in my research on State Aware Imitation Learning (SAIL), reproducing the expert’s observations is key to learning robust behaviors, learning from sparse demonstrations and, of course, to learn from demonstrations where observations is all there is. In this work, we aimed to reproduce the expert’s observations with policies that are too complex for SAIL. Moreover, we wanted to develop an imitation learning approach that is efficient and can learn in real-time on physical systems. We introduced an approach which aims to do just that by using generative models to reason about what to do in any part of the state-space.
This work will be presented at ICLR 2019, you can find the PDF here.
State Aware Imitation Learning
At the core of imitation learning is the question how we may use demonstrations most effectively. We could simply attempt to imitate the expert’s actions if our observations roughly match a demonstrated state but this only works if a demonstration exists that is sufficiently similar. If we have only very little demonstration data, it is highly unlikely that this is the case. In this work, we focussed on reproducing observations that are similar to demonstrations, i.e. observations in which we can know what to do. Reaching arbitrary states in arbitrary domains sounds similar to a reinforcement learning problem and it is thus no surprise to us that the approach we found is based on temporal difference learning, a technique usually reserved for reinforcement learning problems.
This work was presented at NeurIPS 2017, you can find the PDF here.
Directing Policy Search with Interactively Taught Via-points
To develop truly powerful agents, I believe that we will need to develop methods that learn from expert demonstrations, rewards and whatever signal we may get. Either of these signals can be incredibly sparse and hard to learn from on its own. Besides attempting to make a contribution to imitation learning specifically, my thesis research also attempts to keep this in mind. In this work, we developed a combined Reinforcement- and Imitation Learning system which enables the teacher to provide only single observations as demonstrations as well as interactively provide corrections.
This work was presented at AAMAS 2016, you can find the PDF here.
Publications
Refereed Conference Publications
Bullard, K., Schroecker, Y. & Chernova, S. (2019). “Active Learning within Constrained Environments through Imitation of an Expert Questioner.” International Joint Conference on Artificial Intelligence.
Schroecker, Y., Vecerik, M., & Scholz, J. (2019). Generative predecessor models for sample-efficient imitation learning. In International Conference on Learning Representations.
Edwards, A. D., Sahni, H., Schroecker, Y., & Isbell, C. L. (2019). Imitating Latent Policies from Observation. International Conference on Machine Learning.
Schroecker, Y., & Isbell, C. L. (2017). State Aware Imitation Learning. In Advances in Neural Information Processing Systems.
Schroecker, Y., Ben Amor, H., & Thomaz, A. (2016). Directing policy search with interactively taught via-points. In International Conference on Autonomous Agents & Multiagent Systems.
Other Refereed Publications
Edwards, A. D., Sahni, H., Schroecker, Y., & Isbell, C. L. (2018). Imitating Latent Policies from Observation. ICML Workshop on Prediction and Generative Modeling in Reinforcement Learning.