Computer vision technology has come on in leaps and bounds in recent years, enabling machines to get a whole lot better at processing visual data so they can pick out specific objects from background noise, Terminator style. However crunching all that real-time visual data can require a lot of processing power. So, for lighter-weight projects, here’s a neat alternative which allows you to build a bot or system that can identify and track objects by sensing specific colours.
Pixy is an open source vision sensor that’s already exceeded its Kickstarter funding goal of $25,000. It’s able to be fast, says its creators (Charmed Labs and Carnegie Mellon University), because of this hue-centric approach — allowing it to send only specific coloured object visual data to the microcontroller it’s being used with, rather than outputting all visual data for image processing.
The vision sensor can track seven different colours signatures at once — a figure that can be expanded by using combinations of colours (‘colour codes’ — basically sticking one swatch of colour next to a different one to create a combination of two or more colours) if you need to track more than seven different-coloured objects at once. The system can apparently plot and report the real-time location of “hundreds” of objects at a time, and do so at an impressive 50Hz.
Pixy processes an entire 640×400 image frame every 1/50th of a second (20 milliseconds). This means that you get a complete update of all detected objects’ positions every 20 ms. At this rate, tracking the path of falling/bouncing ball is possible. (A ball traveling at 30 mph moves less than a foot in 20 ms.)
To teach Pixy a coloured object to track you hold the object in front of it and press a button on the top of the board which tells it to generate a statistical model of the shades in that particular object. Once it’s done this the board’s LED changes to a correspondingly similar colour and Pixy can start tracking objects of that colour.
Pixy reports objects’ real-time location through one of several interfaces: UART serial, SPI, I2C, digital out, or analog out. There’s also a PixyMon PC or Mac app for viewing what the sensor can see in real-time and for debugging any apps you’re building with it.
To expand function beyond object-sensing, Pixy integrates with an Arduino or other type of microcontroller — so it can be incorporated into another project, such as a wheeled robot — as demoed in the above video. Its creators note that it could also be used to create an indoor navigation system for a robot by, for instance, marking doorways with colour codes so Pixy is able to identify exit/entrance points and tell the robot where to move.
Most of the early bird Kickstarter pledges for Pixy have gone already but $59 can still bag you the vision sensor board plus an Arduino cable — so you can start building your own colour-sensitive bot.