More sessions of the narrative audio AR experience Valtakunnan liitto — Lisätty todellisuus, poistettu totuus? have been scheduled in November, following strong public...
The Full-AAR project
Full-AAR is a research and artistic project exploring the narrative possibilities of audio augmented reality (AAR or Audio AR). In AAR, reality is enhanced with virtual sounds embedded in the environment. In some applications, real-world sounds can also be removed, replaced, or enhanced, such as isolating speech from background noise.
Currently, the project continues through Valtakunnan liitto, an experience that immerses audiences in a biofictional story through narrative and interactive story fragments, set in a real-world location in Helsinki, Finland. The experience highlights Full-AAR's focus: headphone-based AAR using the six-degrees-of-freedom (6DoF) principle, which lets users move and turn their heads freely while virtual sounds remain fixed in three-dimensional space. To learn more about AAR with 6DoF, check the page What is 6DoF AAR?.
To read about recent events related to the Full-AAR project, check the News.
The non-profit project is run by WHS, a contemporary circus and visual theatre group based in Helsinki, Finland. From 2021 to 2025, the project was financially supported by the European Union's NextGenerationEU fund. The aim has been to develop AAR as a storytelling medium with its own narrative language and to share these insights with a wider community interested in narrative uses of AAR. As part of the project, an introductory book on AAR was written: Harju, M. (2025). Audio Augmented Reality: Concepts, Technologies and Narratives. Routledge.
The project places particular emphasis on indoor experiences in complex, multi-room environments, including historical buildings, house museums, and heritage sites. Another focus area is group-oriented applications, enabling multiple simultaneous users to communicate with each other and maintain situational awareness while being acoustically immersed in the narrative.
To the project team’s knowledge, no comprehensive technical platforms or artistic tools for the medium are available. A handful of proprietary technical solutions exist that enable 6DoF AAR applications, but these are not open to the public. Consequently, to run our experiments, the team has constructed their own prototype platform. This work is based on extensive testing of several available virtual audio solutions, indoor tracking systems, game engine workflows, system configurations, and narrative techniques. The resulting prototype enables 6DoF AAR experiences for two or more simultaneous users in multi-room indoor spaces.
Further, to maximise the quality of acoustic illusion while keeping the system convenient to setup and operate, a custom virtual audio solution was developed in collaboration with Aalto University Acoustics Lab. This work resulted in a proof-of-concept application and a preliminary plugin for the audio middleware platform Wwise, but additional development and validation would be required to make the solution publicly available.
Focus areas
In the project, the team is particularily interested in the following topics:
- Narrative possibilities of AAR, mainly utilising 6DoF
- What kind of stories would benefit this medium?
- Identifying and testing characteristic narrative techniques in practice
- Interaction between users; Letting simultaneous users experience the same story with different narrative viewpoints and alternate audio content
- Interplay between real and virtual
- Plausibility of virtual characters when portrayed through different spatial and contextual modes (invisible/acousmatic, attached to an object, heard behind a visual obstacle, etc.)
- Role of interaction and multimodality in supporting the narrative immersion despite any imperfections in the virtual acoustic rendering
- Virtual audio
- Finding and developing the most optimal spatialisation and auralisation solution to enable plausible acoustic illusions while keeping the computational demands convenient
- Pose tracking
- Experimenting with various 6DoF indoor tracking methods suitable for temporary installation in complex multi-room venues
- Using body tracking for kinaesthetic interaction
- Workflows and best practices
- Finding useable and fluent workflows and methods for content-creation
Schedule
The Full-AAR project started in September 2021 and ran as a full-scale research and development effort until summer 2025. During this period, we tested indoor tracking systems and virtual audio technologies, eventually building a prototype 6DoF AAR setup. At the same time, countless narrative ideas were developed, and we explored storytelling techniques suited to the medium.
One major outcome of this work is Valtakunnan liitto (The Reign Union), a bio-fictional interactive story now running at WHS’ gallery space in Helsinki. The experience, which opened to the public in May 2025, draws from the site’s layered history, including a laundry whose owner was linked to an underground Nazi group during World War II. Two participants navigate the same space and share the same storyline—but do they truly hear the same narrative? Who can be trusted, and can you trust what you hear?
Although the research phase has concluded, the project continues through these performances. The team invites visitors to experience the work and share their thoughts on the setup, the narrative, and the sense of immersion.
Team
Matias Harju – Project lead, sound and narrative design, programming, research
Ville Walo – Producing, narrative and concept design, adminstration
Miranda Kastemaa – Lead programming
Teodors Kerimovs – Virtual audio development and research (2023)
Mikko Honkanen — Programming (2022)
Anne Jämsä — Background story research
Contact
For any questions on the Full-AAR project, please drop a message to Matias:
All photos and diagrams on these pages by authors unless credited otherwise.