Real-Time Eye Tracking with JavaScript: Enhancements and Challenges
In this video, I delve into the intricate process of webcam-based eye tracking using JavaScript and TensorFlow.js. I explain the technical details of detecting and processing eye movements by detecting pupils through the BlazeFace model, previously shifting from MediaPipe in Python to a lighter, browser-based implementation. Throughout the video, I actively refactor the code, removing redundancies and explaining our shift from backend to a purely frontend approach. I also touch on the challenges of detecting eye blinks and pupil tracking under various lighting conditions, and how these can be adjusted via the interface.
Try the tool: https://bionichaos.com/EyeTracker
The tools I develop are available on https://bionichaos.com
You can support my work on / bionichaos
#EyeTracking #JavaScript #TensorFlowJS #BlazeFace #WebcamTracking #CodeRefactoring #UIUXDesign #PupilDetection #BiomedicalData #RealTimeProcessing
00:00:00 - Introduction to Webcam-Based Eye Tracking
00:00:12 - Detecting Pupils with TensorFlow.js
00:01:00 - Accessing Public Biomedical Datasets
00:01:30 - Code Walkthrough: HTML and JavaScript Integration
00:02:15 - Issues with Eye Tracking Accuracy
00:03:00 - Refactoring Main JavaScript Code
00:04:17 - Misleading Terminology in Eye Tracking
00:05:25 - Debugging One Eye at a Time
00:06:01 - Feedback on Code Cleanup in Production
00:07:01 - Enhancing Pupil Detection with Grayscale Images
00:08:03 - Overlay Functionality in the UI
00:09:04 - Working with CSS and HTML Adjustments
00:10:04 - Issues with Eye Tracking Display Values
00:11:09 - Adding New Features and Controls to the UI
00:12:05 - Troubleshooting JavaScript Predictions
00:13:06 - Optimizing Canvas Performance in JavaScript
00:14:00 - Final Adjustments and Split View Feature Testing
00:15:01 - Discussing Overlay and Split View Default Settings
00:16:03 - Requests for Viewer Feedback on Code
00:17:08 - Detailing Pupil Detection Functionality
00:18:02 - Version Control Importance
00:19:13 - Preparing Code for Publishing
00:20:01 - Explanation of Project File Structure
00:21:07 - Deep Dive into Pupil Detection Code
00:22:02 - Adjusting Detection Settings Live
00:23:10 - Implementing User Control over Detection Parameters
00:24:06 - Discussion on Algorithm Improvements
00:25:09 - Incorporating User Feedback in Real Time
00:26:12 - Comparing Python and JavaScript Implementations
00:27:09 - Enhancing Algorithm Accuracy
00:28:10 - Final Thoughts on UI and Control Adjustments
00:29:08 - Where to Place New Code Features
00:30:07 - Shortcut Challenges in Development Environment
00:31:08 - Further Refinements on Eye Tracking Accuracy
00:32:08 - Concluding Remarks and Next Steps
00:33:51 - Integration of New Algorithm Features
00:34:10 - Updating HTML and Main JavaScript for New Features
00:35:02 - Final Debugging and Testing of UI
00:36:10 - Implementing New UI Controls for Better Accuracy
00:37:09 - Adding Dynamic Input for Detection Parameters
00:38:32 - Recap and Call for Viewer Suggestions
00:39:41 - Reviewing Updated Eye Tracking Results
00:41:05 - Wrap-Up and Future Plans