“Digital Performance+BCI” System
Paper “Enhancing Audience Engagement in Performing Arts Through an Adaptive Virtual Environment with a Brain-Computer Interface”
Audience engagement is an important indicator of the quality of the performing arts but hard to measure. Psychophysiological measurements are promising research methods for perceiving and understanding audience’s responses in real-time. Currently, such research are conducted by collecting biometric data from audience when they are watching a performance. In this paper, we draw on techniques from brain-computer interfaces (BCI) and knowledge from quality of performing arts to develop a system that monitor audience engagement in real time using electroencephalography (EEG) measurement and seek to improve it by triggering the adaptive performing cues when the engagement level decreased. We simulated the immersive theatre performances to provide audience a high-fidelity visual-audio experience. An experimental evaluation is conducted with 48 participants during two different performance studies. The results showed that our system could successfully detect the decreases in audience engagement and the performing cues had positive effects on regain audience engagement. Our research offers the guidelines for designing theatre performances from the audience’s perception.
Recent development in the BCI techniques has enabled us to measure the audience engagement with off-the-shelf devices with reasonable accuracy. In addition, more and more stage designers have adopted digital performance technique to improve their design. The digital performance technique provides the stage designer a virtual environment in which they can create, manipulate, and test the performance elements in a totally interactive way.
We seek to improve audience engagement by proposing an interactive design tool that measures the audience engagement and adaptively creates immersive theatre performance experience in a virtual environment in real-time. To do that, we propose a novel approach that combines neuroscience and theatre performing arts. We measure the audience engagement in real-time using neural signals obtained from an off-the-shelf wireless EEG headset. The immersive theatre performance experience is generated by a game engine and can be adjusted interactively (Concept Figure).