Robotic Caregivers – Fall 2021

BMED 4833 ROB & BMED 8813 ROB at Georgia Tech

Lectures: Monday & Wednesday, 11 am- 12:15 pm, Whitaker Room 260

Overview

Robotics researchers and futurists have long dreamed of robots that can serve as caregivers. In this project-based course, you’ll learn about future opportunities and present realities for robots that contribute to caregiving. You’ll gain hands-on experience with teleoperation, autonomy, perception, navigation, manipulation, human-robot interaction, and machine learning. You’ll also learn about robot design, healthcare robotics, and entrepreneurship.

Instructor

Prof. Charlie Kemp

Conflict of Interest Statement: This class uses the Stretch RE1 robot from Hello Robot Inc. In addition to being an associate professor at Georgia Tech, Dr. Kemp is a co-founder and the chief technology officer (CTO) of Hello Robot Inc. where he works part time. He owns equity in Hello Robot and is an inventor of Georgia Tech intellectual property (IP) licensed by Hello Robot. Consequently, he receives royalties through Georgia Tech for sales made by Hello Robot. He also benefits from increases in the value of Hello Robot.

Project Videos

At the end of the term, teams will have the option to post their project videos here.

Course Materials

A Project Robot for Each Team

The three project robots in their boxes ready to be opened by the teams.

Each team has exclusive use of a Stretch RE1 mobile manipulator throughout the term that they can access outside of class. This promotes a sense of ownership, including responsibility and care, and avoids the potential for one team to break another team’s work. It also enables them to add custom hardware to their robots and leave it on. This approach also helps with management of the robots, since instructors can wipe and refresh the robots before next year’s class.

To better work together as a team, each team has its own 32″ monitor that they connect to their robot with an HDMI cable. The large size enables multiple team members to work with the robot at the same time. Using a smaller monitor proved difficult during an early programming exercise. Specifically, each team is using a Samsung S32D850T LCD Monitor, which has a resolution of 2560 x 1440 (WQHD). Each team also has a wired keyboard and mouse that they connect to their robot to avoid the potential for interference.

Classes

  • Class 1
  • Class 2
    • “From One to Many: My Personal Quest for Meaningful Mobile Manipulation”
      by Prof. Kemp [Video] [Google Slides] [PDF]
    • Activities
      • Broke up into teams to achieve comparable mixes of skills and degree levels
      • Team members got to know each other with icebreaker questions
      • Each 5-person team spent 10 minutes teleoperating the Stretch RE1 using the game controller with each team member getting a turn
    • Assignments
      • Carefully read the Problem Statement
      • Propose 3 to 5 tasks for your team’s midterm project and provide evidence for their merit
  • Class 3
  • Class 4
    • “Tasks & Teleop Exercise” by Prof. Kemp [Google Slides] [PDF]
    • Activities
      • Each team
        • 30 minutes
          • Discussed tasks proposed by the team members
          • Selected top three proposed tasks
        • 15 minutes
          • Implemented and tested their teleop code on the robot
          • All of the teams succeeded!
        • 5 minutes
          • Presented top three tasks to the entire class
          • Received feedback on their tasks from Prof. Kemp
      • While two teams discussed tasks, the third team worked with the robot
    • Assignments
      • Get started on midterm proposal slides
      • Prepare project related questions for the expert guest lecturers
  • Class 5
  • Class 6
  • Class 7
    • “Evaluating Assistive Devices” by Henry Evans [Video]
    • Discussion with Henry & Jane Evans
      • Henry and Jane Evans are recognized experts on robots for people with disabilities. They have unmatched experience using assistive robots in their home, and a passion for improving the lives of people with disabilities. This was a deeply meaningful guest lecture and an excellent opportunity to benefit from their insights.
  • Class 8
    • Each team worked on its midterm proposal for the next class.
    • Prof. Kemp spent approximately 18 minutes with each team to answer their questions and provide guidance.
  • Class 9
  • Class 10
    • “Getting to Know Your Robot” by Prof. Kemp [Google Slides][PDF]
    •  Activities
      • Each team discussed the feedback they received for their proposal presentation and Prof. Kemp spent time with each team to discuss the feedback
      • Teams began unboxing their robots!
        • The three project robots in their boxes ready to be opened by the teams.
        • To better work together as a team, each team has its own 32″ monitor that they connect to their robot with an HDMI cable. Specifically, they’re using a Samsung S32D850T LCD Monitor, which has a resolution of 2560 x 1440 (WQHD).
        • Note for next time: Leave more time so that each team can unbox and teleoperate the robot using the game controller interface
  • Class 11
    • “Getting to Know Your Robot – Part 2” by Prof. Kemp [Google Slides][PDF]
    •  Activities
      • Teams began working with their robots
      • Teams also carefully stored their robots for the next class
    • Assignments
      • Gain access to the room, so that your team can work with its robot outside of class
      • Begin implementing projects
  • Class 12
  • Class 13
    • The class met via MS Teams.
    • After a brief full-class discussion, the teams went to their own video conference meetings (breakout groups).
      • Prof. Kemp emphasized that each team owns its robot over the term, which implies that each team has freedom in how it works with its robot and the responsibility to take care of its robot.
      • During the full-class discussion, class members raised issues and solutions that they thought might be relevant to other teams. This was a welcome development, since the teams are supposed to be collaborating rather than competing.
    • Prof. Kemp then spent time with each group in the reverse order as the previous class.
      • He let teams know up front that it was fine to not take advantage of his visit and instead focus on working together.
      • Prof. Kemp focused on discussing any questions the team had.
      • Prof. Kemp also asked teams if they were happy with the class and if they were having fun. He encouraged class members to let him know if they had feedback or ideas for the class.
  • Class 14
    • Teams worked with their robots to implement their projects.
    • One team wanted to send commands to their robot from ROS.
    • Another team was attempting to install the basic web interface on GitHub. They were encountering difficulties. One of the other teams had successfully installed it and overcome challenges that occurred. A representative from that team began helping them and volunteered to meet with them outside of class.
    • The third team took their robot to a different location so that they could use the game controller to teleoperate it to perform manipulation tasks with a special object and test out a custom 3D-printed attachment they developed.
  • Class 15
    • Prof. Kemp spent time with each team.
    • He got to try out a team’s systems, which enabled him to provide detailed feedback from the perspective of both an expert and a naive user. He was impressed! The demonstration was the most compelling autonomous interaction he had personally experienced with the Stretch RE1.
    • He visited another team, which was working in another room better matched to their project. He learned more about the team’s approach and difficulties. He also let them know that the default force limit on the arm was likely creating challenges for them. The team had come up with a creative solution to their problem that involved minor modification to the environment.
    • He encouraged a team to be critical of their own work and actively look for weaknesses. He emphasized that unlike research and business in the real world, this class encourages negative results!
  • Class 16
    • Due to ROS World 2021, Prof. Kemp was unable to attend class. Teams worked on their own preparing for their midterm project presentations. The following day, Prof. Kemp held open office hours via MS Teams. Only one team attended.
  • Class 17
    • Each team presented their midterm project for 10 minutes and then had 10 minutes to answer questions from a panel of experts. Questions were restricted to the panel of experts and graders. The next class will be devoted to in-class discussion about each of the projects, so all class members will have time to ask questions.
    • Expert Panel
      • Henry and Jane EvansDr. Vy Nguyen, and Julian Cheng served on the expert panel via MS Teams on a Linux laptop connected to a large monitor, a sound bar, and a high-quality wide-angle webcam placed in the middle of the room.
      • Julian Cheng added his expertise on commercialization, startups, and fundraising to the panel. He is a venture capitalist (VC) and the Founder and Managing Partner of Gen 1 Capital, which is an investor in Hello Robot. Julian is an engineer by training, active in higher education, and familiar with the Stretch RE1.
      • Panelists were informed ahead of time that they need to consider the teams’ presentations as confidential, since the teams own any intellectual property (IP) they create.
    • Prof. Kemp, Henry Clever, and Patrick Grady graded the presentations using the midterm project rubric.
    • All of the projects were distinct and exciting! Each team showed a compelling video and one of the teams gave a successful live demonstration!
  • Class 18
    • “Final Project Discussion” by Prof. Kemp
    • Activities (see Prof. Kemp’s slides for additional details)
      • Team members read the graded rubrics that were posted in the morning.
      • Prof. Kemp provided general feedback on the midterm projects.
      • The class discussed the final project and final project proposal.
        • Final project video
          • Students were in favor of each team creating a short video (30 sec to 1.5 min)  suitable for posting on the Internet.
          • Students were also in favor of Prof. Kemp actually posting the three videos, but they will have the option to change their minds.
          • Deciding on this now gives teams a clear goal and a better chance of creating videos that they’re happy to share with the world.
        • Students were in favor of Prof. Kemp creating a new final proposal rubric even though there is not much time until their presentations.
      • For the last 30 minutes, the class discussed each team’s project for 10 minutes with an emphasis on next steps. This was enjoyable and highly valuable!

Credit

Zackory Erickson

As a graduate student, Prof. Zackory Erickson co-developed and co-taught the original version of this course with Prof. Kemp  in spring 2020 and spring 2021. The original version of this course focused on projects using Assistive Gym, a physics simulation environment for assistive robotics. Prof. Zackory Erickson is now a tenure-track faculty member in the Robotics Institute at CMU.

Affiliations

Previous Terms

2020 Spring

2021 Spring