The Full-AAR Project

Full-AAR is an artistic research project by WHS Theatre Union, a contemporary circus and visual theatre group based in Helsinki, Finland. The project explores the narrative possibilities of audio augmented reality (AAR) with six degrees of freedom (6DoF).

This non-profit project aims at developing 6DoF AAR as a storytelling medium with its own narrative language. The emphasis is on indoor experiences in complex multi-room environments such as historical buildings, home museums, etc.

Since no comprehensive technical platforms or artistic tools yet exist for the medium, a number of virtual audio solutions, indoor tracking systems, game engine workflows, system configurations, and narrative techniques have been tested since the start of the project in autumn 2021. Based on that work, a prototype setup has been constructed enabling 6DoF AAR experiences for two or more simultaneous users in a multi-room indoor space.

Further, to maximise the quality of acoustic illusion while keeping the system convenient to setup and run, a custom virtual audio solution is being developed in collaboration with Aalto University Acoustics Lab.

These pages aim to provide detailed information on the Full-AAR project with insights and recommendations on creating 6DoF AAR experiences. The content is still very much a work in progress and will be constantly updated with new stuff as the project progresses.

The Full-AAR project is financially supported by the European Union NextGenerationEU fund.

In the project, we are particularily interested in the following topics:

• Narrative possibilities of 6DoF AAR
  • What kind of stories would benefit this medium?
  • Identifying and testing characteristic narrative techniques in practice with narrative demos
  • Letting simultaneous users experience the same story with different narrative viewpoints and alternate audio content

• Virtual audio
  • Finding and developing the most optimal spatialisation and auralisation solution to enable plausible acoustic illusions while keeping the computational demands convenient

• Positional tracking
  • Experimenting with various 6DoF indoor tracking methods suitable for temporary installation in complex multi-room venues
  • Using body tracking for kinaesthetic interaction

• Workflows and best practices
  • Finding useable and fluent workflows and methods for content-creation

The project started in September 2021, and with the current EU funding, the project will last at least until the end of 2023. The first 1.5 years have been testing different technologies for indoor tracking and virtual audio. Simultaneously, we have been brainstorming with countless narrative ideas and writing scenes for the first demo experience. (You can check the other pages on this wiki for more about our efforts and findings in positional tracking, virtual audio, narrative exploration, system architecture, etc.)

Now, as the technical setup is about to work as intended, enabling a more or less smooth 6DoF AAR experience, we have been concentrating more on the first demo to be premiered this year (2023). The first dialogue recordings have been conducted in November 2022, and they will still continue during this spring. Narrative scripting in game engine and sound design work are getting more intense. We are also starting to make tests with two simultaneous users hearing the story from alternate points of view.

We are also inviting visitors and different audiences to test and discuss the setup, narrative content, and level of immersion before the actual launch of the demo.

We hope to be able to continue the project even after this year, aiming at more demos in various locations with interesting stories to be told.

Matias Harju - design, research, story, programming
Ville Walo - design, story, narrative research, administration
Miranda Kastemaa - programming
Teodors Kerimovs - virtual acoustics research
Mikko Honkanen - programming
Anne Jämsä - narrative research

For any questions on this wiki or the Full-AAR project, please drop a message to Matias: