Researchers from MIT, Microsoft Research, and Adobe Research have been collaborating on an intriguing project. They are experimenting with recovering sound from high-speed video of the minute vibrations from objects. The project is known as The Visual Microphone, and according the project website, this is how it works:
When sound hits an object, it causes small vibrations of the object’s surface. We show how, using only high-speed video of the object, we can extract those minute vibrations and partially recover the sound that produced them, allowing us to turn everyday objects—a glass of water, a potted plant, a box of tissues, or a bag of chips—into visual microphones. We recover sounds from highspeed footage of a variety of objects with different properties, and use both real and simulated data to examine some of the factors that affect our ability to visually recover sound. We evaluate the quality of recovered sounds using intelligibility and SNR metrics and provide input and recovered audio samples for direct comparison. We also explore how to leverage the rolling shutter in regular consumer cameras to recover audio from standard frame-rate videos, and use the spatial resolution of our method to visualize how sound-related vibrations vary over an object’s surface, which we can use to recover the vibration modes of an object.
You can see the (surprisingly accurate) results in the video above for yourself- the experiment using a normal DSLR is particularly extraordinary.
You can find more example experiments here, or read the research paper for yourself here.
(Video and featured image credit: The Visual Microphone)
Interested in more content like this? Sign up to our newsletter, and you wont miss a thing!
The Author
