S  | SEEING THROUGH SOUND

Sound in 3D coordinate space serves as the foundation for how this sonic language behaves. Tone with a wah-wah effect, pitch, and volume correlate to width, height, and depth respectively.
Distinct sound effects are added to identify materials, colors, textures, mass, or content. Each aspect of the language takes advantage of our learned associations and natural intuitions, such as bass for heaviness or noise for rough materials.
Each platform uses capacitive sensing to trigger sounds when approached by an individual.
This final prototype demonstrates how an object may sound like with 3D shape, material, and color information embedded in a reactive soundtrack. As the hand moves around an object, the base soundtrack transforms based on the palm's position in relation to the object in space. Conducted through Leap Motion and MaxMSP.
Test subjects were taught what each axis or color sounds like and tested on whether they could identify shapes in each plane while listening to pre-recorded soundtracks.
Final sound designs were tested by presenting platforms with different colors, textures, and boxes with different weights.
Show More

 

 

Inspired by the notion of inducing synesthesia, S3 is an experimental investigation of how we may improve our less dominant senses by adding extra layers of information. What if we can use sound to enhance perceptions that usually depend on vision? What if hearing a specific sound immediately creates a mental picture of the object it represents?

 

S3 is a sonic language for objects. To define a shape in space, basic sound elements of tone, pitch, and volume correlate to width, height, and depth respectively. Distinct sound effects are added to identify materials, colors, textures, mass, or content. Each aspect of the language takes advantage of our learned associations and natural intuitions. Ultimately, S3 provides a preview of a future where we can more directy link vision and sound to redefine imagination and perception of the world around us.

 

______

 

- Exhibited in Sharing Design: Culture of Utopia Makers -
La Fabbrica del Vapore, Milan Design Week 2015