Student Work: AI-based computer interface for enhanced accessibility
Henri Sayag, sophomore at Kehillah High School
Abstract
While volunteering at AbilityPath, a non-profit organization dedicated to providing services to people with disabilities, I realized a major problem some people faced was not access to technology but being able to use it. Specifically, the high levels of fine motor control needed to use the interface (such as mice, trackpads, keyboards, and game controllers) prohibit many people’s access. This problem led me toward a project to enhance computer accessibility without increasing cost.
I aimed to simplify and make computer interfaces more natural and accessible by using a much more common and direct form of communication: gestures. Using (I show how to get) signals captured by the webcam, I will show how the data is processed before being fed through two major AI models to first extract hand-landmark positions and then gestures from the landmarks.
Although existing commercial methods using facial expressions captured by a camera to control a game do exist, there is an overwhelming population of people who have troubles with fine motor control and won’t benefit from an unintuitive, fairly unnatural form of the game interface. I minimized the required technology to further universalize the access, by using open-source software and free tools, as well as making the entire project open-source and thus free.
Bio:
Henri Sayag, a sophomore at Kehillah High School, is interested in CS, AI, and mathematics. He’s a member of USACO and volunteers at a local non-profit serving people with developmental disabilities. He is an avid guitarist, sailor, and potter. Recently he has been experimenting with R-NNs and Neural Networks at a low level, as well as learning CUDA C for multiprocessing on specific hardware.
0:00 Chapter Introduction and Announcements
2:16 Speaker Introduction
3:40 Presentation
41:15 Q & A