Explore the connections between sound and space in Augmented Reality
Roles:
- Conception and leading the project in design and development.
- Through rapid-prototyping in Unity, we were able to quickly test new interaction principles in hands-on UX testing of a magic window VR environment.
- Directed video documentation and composed original soundtrack.
- Involved with the project from conception to completion.
- Invited to represent Left Field Labs in the Google booth at GDC '15 to share our VR apps with thousands of conference attendees.
The concept for Soundfield was inspired by my previous experience in a funded media arts research group where I created sound pieces based on audio-visual synesthesia and feedback loops.
Soundfield builds upon the Bouba / Kiki experiment from 2001 in which users assigned the names “Bouba” and “Kiki” to two distinct line drawings. Despite having participants from different cultural and linguistic origins, the two subject groups in the experiment assigned with 95-98% consistency that the rounded shape was named “Bouba” and the spiky shape was named “Kiki”. This outcome suggests that the naming of objects, and our understanding of the connections between sound and form are not entirely arbitrary.
In Soundfield we sought to further explore these innate ties between sight and sound by extruding audio reactive forms, similar to those from the original experiments, into three dimensional space.
Full range motion tracking and spatial awareness from the device allows users to navigate the landscape and go over, under, between and through an orchestra of animated sound-objects. These illustrative forms embody spatial features directly reflective of the sound characteristics they emit, and are fully explorable using Project Tango technology.
