Classification of Household Materials via Spectroscopy

Paper | IEEE

Z. Erickson, N. Luskey, S. Chernova, and C. C. Kemp, “Classification of Household Materials via Spectroscopy”, IEEE Robotics and Automation Letters (RA-L), 2019. (presented at ICRA 2019) (Finalist for Best Paper Award in Service Robotics at IEEE Conference on Robotics and Automation (ICRA 2019))

Abstract

Recognizing an object’s material can inform a robot on the object’s fragility or appropriate use. To estimate an object’s material during manipulation, many prior works have explored the use of haptic sensing. In this paper, we explore a technique for robots to estimate the materials of objects using spectroscopy. We demonstrate that spectrometers provide several benefits for material recognition, including fast response times and accurate measurements with low noise. Furthermore, spectrometers do not require direct contact with an object. To explore this, we collected a dataset of spectral measurements from two commercially available spectrometers during which a robotic platform interacted with 50 flat material objects, and we show that a neural network model can accurately analyze these measurements. Due to the similarity between consecutive spectral measurements, our model achieved a material classification accuracy of 94.6% when given only one spectral sample per object. Similar to prior works with haptic sensors, we found that generalizing material recognition to new objects posed a greater challenge, for which we achieved an accuracy of 79.1% via leave-one-object-out cross-validation. Finally, we demonstrate how a PR2 robot can leverage spectrometers to estimate the materials of everyday objects found in the home. From this work, we find that spectroscopy poses a promising approach for material classification during robotic manipulation.

Code

https://github.com/Healthcare-Robotics/smm50

SMM50 Dataset

SMM50 dataset (16 MB): https://goo.gl/Xjh6x4

cd smm50/data
wget -O smm50.tar.gz https://goo.gl/Xjh6x4
tar -xvzf smm50.tar.gz
rm smm50.tar.gz

Object List

Metal	Plastic	Wood	Paper	Fabric
steel aluminum aluminum foil copper magnesium titanium brass lead zinc iron	PP PVC HDPE PET blue polyethylene green polyethylene red polyethylene yellow polyethylene thermo polypropylene thermo teflon	ash cherry curly maple hard maple hickory red cedar red elm red oak walnut white oak	cardboard green construction paper orange construction paper red construction paper magazine paper newspaper notebook paper printer paper receipt paper textbook paper	cotton canvas cotton sweater cotton towel denim felt flannel gauze linen satin wool