Terrapattern is reverse image search for maps, powered by a neural network


Terrapattern is a visual search engine that, from the first moment you use it, you wonder: Why didn’t Google come up with this 10 years ago? Click on a feature on the map — a baseball diamond, a marina, a roundabout — and it immediately highlights everything its algorithm thinks looks like it. It’s remarkably fast, simple to use and potentially very powerful.

Go ahead and give it a try first to see how natural it is to search for something. How does that work? And how did a handful of digital artists and developers create it — and for under $35,000?

The secret, as with so many other interesting visual computing projects these days, is a convolutional neural network. It’s essentially an AI-like program that extracts every little detail from an image and looks for patterns at various levels of organization — similar to how our own visual system works, though the brain is infinitely more subtle and flexible.

In Terrapattern’s case, the neural network was trained to look at small squares of the landscape and, comparing those patterns to a huge database of tagged map features from OpenStreetMap, it learned to associate them with certain concepts.

Think of how a camera recognizes a face and knows when it is blinking or smiling. It doesn’t actually “know” what faces, smiles and eyes are, but it associates them with certain patterns of pixels, and can reliably pick them out.

terrapattern patterns

Once Terrapattern had been trained to recognize and categorize all manner of geographical features, from boats to water towers, its creators set it free on detailed maps of the greater New York, Pittsburgh, Detroit and San Francisco areas. It scoured the landscape and built a huge database of features and similarities — which can be quickly queried and the results returned immediately (the neural network isn’t doing any “thinking” when you click on a feature — its work is done for this dataset).

Of course, you could just search for “tennis fields in Oakland” or the like and get perfectly good results, but this allows one to search for things that may not be listed so formally. What if you were looking for houses in the middle of fields, or cul de sacs, or dead lawns, or circular parking lots? Terrapattern knows where those are just as much as it knows where the airports and ferry terminals are. They’re all just assemblages of features to the neural network.

Terrapattern was made by Golan Levin, David Newbury and Kyle McDonald, with money from the Knight Foundation’s Prototype Fund. With the resources they have, they were able to map the four cities mentioned, but more are coming soon. And with luck, feature detection at higher and lower levels. It’s easy to find a ballpark, but hard to find, say, four-way stops (at the small level) or prison complexes (at a larger one).

The work is free under a Creative Commons 4.0 license, and you can check out their code over at GitHub.