Immersive Audio Guiding System (IAGS)

About the research project

The aim of the "Immersive Audio Guiding System (IAGS)" research project funded by innosuisse was to develop a scalable and easy-to-configure mixed reality guiding system for indoor spaces.

Originally, audio guides only served one purpose: to convey information. A voice guides visitors through an exhibition and provides facts about the exhibits they are showing interest in. Only in rare cases are guides thematically or artistically designed in any way. The IAGS intends to fill this gap and to enable the design of highly sensual and interactive exhibitions. 

With the help of an indoor tracking system, the exact position and head orientation of exhibition visitors can be continuously determined indoors. Sounds can be placed as virtual audio sources at specific locations in the room. Depending on the individual position and head orientation of the visitors, text, music and/or sound design are played back via headphones and the sounds always come from the intended direction. The latter is realized in real time using dynamic binaural synthesis, which incorporates both the characteristics of the human ear and acoustic room characteristics (so-called BRIRs). The result is a novel, three-dimensional listening experience.

The audio content is placed and programmed beforehand in a so-called 'digital twin', a digital version of the physical exhibition space. The approach is similar to the creation of a modern computer game. This digital twin is then transferred to the mobile devices and precisely mapped onto the physical space. By interpreting the data provided by the tracking system, the IAGS lets the visitor become the main character in the computer game, so to speak. The high-precision localization of visitors is realized with the help of an ultra-wideband system (UWB) and a head-tracker module integrated into the headphones. 

After testing the IAGS in public for the first time in the Pissarro Sounds exhibition at the Kunstmuseum Basel in 2021, we've had the opportunity to fully exploit its strengths in the Multilingual Switzerland exhibition at the Landesmuseum Zürich in 2023.

Project partner: iart & FHNW 
Research team: Thomas Resch, Hannes Barfuss, Anna Pfeiffer, Elias Hodel, Ramon De Marco, Jascha Dormann, Valentin Spiess, Ivo Schüssler, Christian Studach, Michael Lotz, Marcel Colomb

See also: Second Layer SoundWalk