Lille.Pod

While 3D user interfaces are often designed with efficiency and accuracy in mind, artistic performances have their own very specific constraints and criteria for a successful interaction in mixed or virtual reality, which have yet to be fully understood. In this paper, we study the design of 3D interactions for a Spatial Augmented-Reality (SAR) display in the context of a cyber-opera. We perform an analysis of design decisions taken during the multiple residencies and lab sessions, and we conduct a reflexive thematic analysis of interviews with the director and actors, which highlight how they integrated the story, the actors’ performance, the audience experience and the technology. We identify four main criteria for designing 3D interactions in performing arts, namely their efficiency, expressiveness, contextualization and legibility, and we derive guidelines for future research and creation.

Cagan Arslan, Florent Berthaut. Examining the Design Process for 3D Interactions in Performing Arts : A Spatial Augmented Reality Cyber-Opera Case Study. IEEE Transactions on Visualization and Computer Graphics, 2025, pp.1-10. ⟨10.1109/TVCG.2025.3549194⟩. ⟨hal-04938727⟩

Scientists, artists and performers have explored the relationship between visuals and sound on numerous occasions, to try and find/define ways to fuse both sensory experiences together. Immersive technologies further amplify the possibilities for multimodal expression in performance contexts. While previous frameworks exist for the design of immersive musical instruments and audio-visual instruments independently, we believe that their combination raises specific concerns, which might prevent the exploration of expressive opportunities.In this article, we define the concept of Extended Reality Audio-Visual Instruments (XRAVI) and propose a framework for their analysis and design, based on the literature on audio-visual and extended reality instruments. We evaluate it through a collaborative autoethnographic work, where an initial shared instrument has been developed and practised following two different approaches: audio-first and visual-first. From it, we derive insights and guidelines for the design of XRAVI.

Esther Gruy, and Florent Berthaut. 2026. Extended Reality Audio-Visual Instruments: Design Framework and Case Study. Proceedings of the International Conference on New Interfaces for Musical Expression. DOI: 10.5281/zenodo.20784281 [PDF]

Intervention d'Hélène Delage, de l'Université de Genève, sur l'évaluation dynamique en orthophonie, dans le cadre du Séminaire APPEL, organisé par le laboratoire STL.