CP2: Object Detection – Webcam Tracking in p5.js/TensorFlow
In the past few years, we've seen a huge jump in performance and availability of new tools for helping computers see and identify objects in the world. This example uses TensorFlow.js, the Javascript version of Google's widely-used (and suuuuper complicated) library TensorFlow.
In this example, TensorFlow.js loads COCO (Common Objects in Context), a long-used machine-learning model sponsored by Microsoft and Facebook. It is trained on about 330k images of 91 different objects. For each image, a human drew an outline around that object, which is then fed into a neural network that crunched the pixels from all of those images, resulting in a rather compact 'model' that can be used on images the system has never seen before!
Another benefit to AI models like this is they are really robust in all lighting conditions, unlike our color-tracking code! (Of course, as a trade-off they are way more complicated, may have bias in their training data, and can have a negative environmental impact.)
👩💻 https://editor.p5js.org/jeffThompson/...
ALL THE VIDEOS IN THIS UNIT
🎥 • CP2: Week 04 (Webcam Tracking)
MORE ABOUT COCO:
💡 https://cocodataset.org
BASED ON
💡 https://github.com/tensorflow/tfjs-mo...
CHALLENGES
❓ Can you find the center of an object's bounding box and draw a shape there?
❓ Can you watch for a specific object and use it's x/y (or center) coordinates to display a shape or image onscreen?
❓ Can you create a loading screen while the model is being loaded?