Gesture recognition with ToF sensor
This video shows a demonstration of gesture recognition using a time-of-flight sensor ring.
• Sensor ring comprises 10 VL53L8CX Time-of-Flight sensors capturing 10 depth images of resolution 8x8 at 15Hz
• 3 sensors cover the effective workspace
• Preprocessing: background is removed more than 30cm
• Sliding window of 32 frames (~2s) fed to gesture recognition classifier
• Classifier is a temporal CNN model
• Trained with ~100 sequences depicting 3 gestures: “swipe-up”, “swipe-down” and “approach”
• Output is best label and associated probability (retained if more than 90%)
• Demo GUI that visualizes images and classification result
This work was carried out within the Horizon Europe project Sestosenso (GA 101070310).