New Technique Allows Researchers To Extract Audio From Silent Video

By looking at the minute vibrations in high speed video, researchers at the Department of Engineering of the Catholic University of America have created a method for extracting sound data from high-speed silent video footage.

From the release:

The authors used a subset-based image-correlation approach to detect the motions of points on the surface of an object, capturing target images with a high-speed camera and applying the Gauss-Newton algorithm and a few other measures to achieve very fast and highly accurate image matching. Because the detected vibrations are directly related to sound waves, a simple model was used to reconstruct the original audio information of the sound waves.

While this wouldn’t work for, say, extracting sound from old silent film, the researchers say they could extract conversations from vibrations at a “far distance,” a prospect that is at once fascinating and a bit scary.

The paper, “Audio extraction from silent high-speed video using an optical technique,” appeared in the journal Optical Engineering.

This technique is not new – MIT researchers were able to read sound from a stray potato chip bag – but this method uses a simpler image matching system to sense audio.