The Full-AAR project

Full-AAR is an artistic and tech development project by WHS Theatre Union, a contemporary circus and visual theatre group based in Helsinki, Finland. The project explores the narrative possibilities of audio augmented reality (AAR) with six degrees of freedom (6DoF).

To learn more about the medium, check the page What is 6DoF AAR?.

To read about recent events related to the project, check the News.

This non-profit project aims at developing 6DoF AAR as a storytelling medium with its own narrative language. The emphasis is on indoor experiences in complex multi-room environments such as historical buildings, home museums, heritage sites, etc.

Another focus area is group-oriented applications with multiple simultaneous users being able to communicate with each other and maintain situation awarness while being acoustically immersed in the narrative.

Since no comprehensive technical platforms or artistic tools yet exist for the medium, a number of virtual audio solutions, indoor tracking systems, game engine workflows, system configurations, and narrative techniques have been tested since the start of the project in autumn 2021. Based on that work, a prototype setup has been constructed enabling 6DoF AAR experiences for two or more simultaneous users in a multi-room indoor space.

Further, to maximise the quality of acoustic illusion while keeping the system convenient to setup and run, a custom virtual audio solution is being developed in collaboration with Aalto University Acoustics Lab.

Actor Emma Castrén recording dialogue in an anechoic chamber at Aalto Acoustics Lab

These pages aim to provide detailed information on the Full-AAR project with insights and recommendations on creating 6DoF AAR experiences. The content is still very much a work in progress and will be constantly updated with new stuff as the project progresses.

The Full-AAR project is financially supported by the European Union NextGenerationEU fund .

In the project, we are particularly interested in the following topics:

• Narrative possibilities of 6DoF AAR
  • What kind of stories would benefit this medium?
  • Identifying and testing characteristic narrative techniques in practice
  • Interaction between users; Letting simultaneous users experience the same story with different narrative viewpoints and alternate audio content
  • Interplay between real and virtual
  • Plausibility of virtual characters when portrayed through different spatial and contextual modes (invisible/acousmatic, attached to an object, heard behind a visual obstacle, etc.)
  • Role of interaction and multimodality in supporting the narrative immersion despite any imperfections in the virtual acoustic rendering

• Virtual audio
  • Finding and developing the most optimal spatialisation and auralisation solution to enable plausible acoustic illusions while keeping the computational demands convenient
• Positional tracking
  • Experimenting with various 6DoF indoor tracking methods suitable for temporary installation in complex multi-room venues
  • Using body tracking for kinaesthetic interaction
• Workflows and best practices
  • Finding useable and fluent workflows and methods for content-creation

Teodors measuring acoustic properties of the venue with spatial room impulse responses (SRIRs)

The project started in September 2021, and with the current funding, the project will last at least until the end of 2023. During the initial 1.5 years, we conducted tests on various technologies for indoor tracking and virtual audio. Simultaneously, we have been brainstorming with countless narrative ideas and writing scenes for the first demo experience.

Now, as the technical setup is about to work as intended, enabling a more or less smooth 6DoF AAR experience, we have been shifting our focus a bit more on the first demo to be premiered this year (2023). We are also inviting visitors and different audiences to test and discuss the setup, narrative content, and level of immersion.

We aim to continue the project beyond this year, collaborating with various partners at their respective locations to create 6DoF AAR experiences out of their countless intriguing stories.

Matias Harju - design, research, story, programming
Ville Walo - design, story, narrative research, administration
Miranda Kastemaa - programming
Teodors Kerimovs - virtual acoustics research
Mikko Honkanen - programming
Anne Jämsä - narrative research

For any questions on this wiki or the Full-AAR project, please drop a message to Matias:

All photos and diagrams on these pages by authors unless credited otherwise.