Audio-Visual Object Classification for Human-Robot Collaboration - Alessio Xompero
Audio-Visual Object Classification for Human-Robot Collaboration
Alessio Xompero, Centre for Intelligent Sensing, Queen Mary University of London (QMUL), U.K.
Yik Lung Pang, Centre for Intelligent Sensing, Queen Mary University of London (QMUL), U.K.
Timothy Patten, School of Mechanical and Mechatronic Engineering, University of Technology Sydney (UTS), Australia
Ahalya Prabhakar, Learning algorithms and systems Laboratory, École polytechnique fédérale de Lausanne (EPFL), Switzerland
Berk Calli, Computer Science Department, Worcester Polytechnic Institute (WPI), USA
Andrea Cavallaro, Centre for Intelligent Sensing, Queen Mary University of London (QMUL), U.K.
Presented at the 47th IEEE International Conference on Acoustics, Speech, & Signal Processing (IEEE ICASSP), Singapore, May 22-27, 2022.
Human-robot collaboration requires the contactless estimation of the physical properties of containers manipulated by a person, for example while pouring content in a cup or moving a food box. Acoustic and visual signals can be used to estimate the physical properties of such objects, which may vary substantially in shape, material and size, and also be occluded by the hands of the person.
To facilitate comparisons and stimulate progress in solving this problem, we present the CORSMAL challenge and a dataset to assess the performance of the algorithms through a set of well-defined performance scores. The tasks of the challenge are the estimation of the mass, capacity, and dimensions of the object (container), and the classification of the type and amount of its content. A novel feature of the challenge is our real-to-simulation framework for visualising and assessing the impact of estimation errors in human-to-robot handovers.
Paper: https://doi.org/10.1109/ICASSP43922.2...
ArXiv: https://doi.org/10.48550/arXiv.2203.0...
Poster: https://corsmal.eecs.qmul.ac.uk/resou...
Webpage: https://corsmal.eecs.qmul.ac.uk/chall...