Whenever I try to take a photo through a plane or hotel window, chances are there are plenty of reflections that show up on the final image and ruin it. Now, however, Google and MIT researchers have found a way to take these photos and automatically remove these reflections and other obstructions.
In a paper they will present at Siggraph 2015 later this month, Tianfan Xue, Michael Rubinstein, Ce Liu and William T. Freeman show how you can take a short video sequence with your phone and then let their algorithms do their magic.
Reflections, rain drops and fences mostly disappear from the final image. The algorithms looks at the different images from the video and then figure out what’s an obstruction in the foreground and what’s part of the background. Others have tried this before, but none of their results are as good as what these researcher came up with.
And here is where it gets spooky: because the algorithm has to detect the reflection in a window, for example, it can also create a clear picture of the reflection (or the fence — but most fences I have ever seen aren’t very interesting…). That’s the kind of sci-fi stuff you usually see on bad cop shows (the kind where the technician can always zoom in on a license plate or reflection in a window, no matter the resolution of the image).
Sadly, there’s no word on when this will come to a smartphone camera near you.
To fully appreciate this, check out the video below: