ROS Commander (ROSCo): Behavior Creation for Home Robots

rosco_interface

We introduce ROS Commander (ROSCo), an open source system that enables expert users to construct, share, and deploy robot behaviors for home robots. A user builds a behavior in the form of a Hierarchical Finite State Machine (HFSM) out of generic, parameterized building blocks, with a real robot in the develop and test loop. Once constructed, users save behaviors in an open format for direct use with robots, or for use as parts of new behaviors. When the system is deployed, a user can show the robot where to apply behaviors relative to fiducial markers (AR Tags), which allows the robot to quickly become operational in a new environment. We show evidence that the underlying state machine representation and current building blocks are capable of spanning a variety of desirable behaviors for home robots, such as opening a refrigerator door with two arms (video), flipping a light switch (video), opening a drawer (video), unlocking a door, and handing an object to someone (video). Our experiments show that sensor-driven behaviors constructed with ROSCo can be executed in realistic home environments with success rates between 80% and 100%.

Three Tier of Interfaces Matched to Users’ Expertise

ROSCo—as a system for using, deploying, and creating behaviors—has interfaces at three different levels. The first level, shown in the first figure, is intended for expert users that are willing to spend time learning the interface (perhaps through video online tutorials and how-to guides) but are not necessarily expert roboticists. Behaviors, represented as HFSMs, are constructed at this level using parameterized building blocks where there each block is matched an appropriate graphical interface. Once constructed, they can be saved to disk, reused, and shared.

giving_a_tour
Level 2: Interface for deploying behaviors for computer literate users or agents at a call center (video).

The second tier interface, shown above, is designed for users with an intermediate level of expertise and conceptualized as a process of users giving a tour of their home to the robot, telling it how to operate what and where. This “touring” interface works by asking users to attach an ARToolKit marker, dragging a 3D frame relative to that marker to a behavior specific spot (e.g. the middle of a drawer handle), then selecting the desired behavior.

web_interface
Level 3: Interface for running behaviors (video).

Designed for users that just wants to use the robot with its potentially numerous capabilities, the third interface allow users to start and control ROSCo behaviors. Users start by selecting the desired behavior from hierarchically organized menus, similar to music play lists in an MP3 player, then telling the behavior to start.

General Automation Enabled Opulence

Shown above are ROSCo behaviors for using the PR2 to give a back rub and use a hand fan. Although uncommon in research robotics, we believe such uses might be commonplace once general purpose robots are in homes and users empowered by expert interfaces such as ROSCo to create new robot capabilities. Uniquely, general purpose robots has the potential to displace many smart and motorized devices in homes as well as create new opportunities for automation.

In the BackRub demo (named after a famous search engine), users first activate the behavior through ROSCo’s web interface then lean into the PR2’s grippers for a “back rub.” Stopping the behavior can also be carried out through the web interface (which runs on mobile phones).

With the fanning behavior, users first activate the behavior through ROSCo’s web interface then place the fan in the robot’s gripper, after which the behavior plays a looping motion.

Shared Autonomy Teleoperation for the Motor Impaired

Home environments are complex and varied presenting significant and unsolved challenges to robotics. Tools such as ROSCo can help address these difficulties by producing autonomous capabilities that can be used during shared teleoperation. In such scenarios, robot autonomy reduces the mental demand on users and teleoperator control enable mobile manipulators to operate more robustly in home environments.

We tested our system with Henry Evans, a man with quadriplegia, in his home as part of a teleoperation interface where repeatable actions such drawer opening are performed using ROSCo behaviors, and harder tasks are performed by human teleoperator. Using such systems based on shared autonomy, we hope to restore the ability to live independently to similarly motor impaired individuals.

More Information

For questions, contact either Hai Nguyen or Charles C. Kemp, the authors of this work. The code is available in the packages rcommander_core and rcommander_pr2 at ros.org. We also refer readers to our publications for more information: