Introduction
Today, we zero in on the intriguing intersection between the visual and aural elements of musical performance. Those in the audience at a concert or recital aren’t just listening - they’re also watching. Musicians typically consider sound to be the paramount component of their performances. While sound does form the core, the visual element of a performance also plays a big role. Along with the music, the sight of the performers, and their gestures and movements, create a multi-sensory experience for the audience, and influence how a performance is received.
In an enlightening research project undertaken in 2018, and presented in their article "Animated Performance: ‘Better’ Music Means Larger Motions", Caitlyn Trevor and David Huron explored the visual facets of performance, to investigate the extent to which the conscious use of physical gestures by musicians might enhance the perceived quality of a performance.
By recording musicians' gestures during performances, and translating them into digital animations, Trevor and Huron were able to assess the impact of physical movements on perceived musicality, including so-called 'secondary motions’ i.e. those not essential or relevant to sound production.
The Research
Trevor and Huron used motion capture technology to record 4 different musicians from Ohio State University performing two different solos they were familiar with. One of the passages was slow and lyrical, the other was quicker and more challenging from a technical perspective.
Using motion sensors, the movements made by the musicians while playing were converted into digital animations of their performances. For each original recording, 3 different digital animations were created: one with augmented performance motion, one with the original motion of the musicians, and one with a diminished level of physical motion.
The 3 animations were then combined into single dynamic videos where the range of motion on the animation could be changed - using a simple slider - from diminished performer motion through to original motion, and then to the augmented or exaggerated level of performer motion.
In the second part of the experiment, a group of musicians (consisting of music undergraduates, post-graduates, and PhD holders) were asked to adjust the range of the movements made by the animated figures, while listening to the relevant passages of music, to create what they thought would be the best musical performance in each case. The results were telling.
The Results
Regardless of the tempo or complexity of the performed passages, the participants in the second stage of the experiment consistently enhanced or exaggerated the motions on the digital animations during the performances. In other words, they equated larger, more pronounced physical motions with a more expressive, more musical, better quality performance.
Trevor and Huron also identified three particular musical parameters that seemed to motivate these more expressive physical movements, namely
- an increase in dynamics;
- rhythmic density; and
- the overall directionality of the phrase.
Interestingly, the study also indicated that those participants with less sophisticated musical backgrounds were much more likely to prefer performances that involved greater levels of motion and expression. This suggested to Trevor and Huron that physical expressiveness could be a key tool in broadening the appeal of music performances to wider, more diverse audiences.
Key Learnings
The outcomes of the research are potentially very valuable for performers and music educators. Musicians are not just creating sound when they perform - the are crafting a comprehensive, multi-modal experience for their audience. So the more we understand and acknowledge the connections between aural and visual information, the more convincing and engaging our performances can become.
In addition, knowing which musical features inspire or restrict secondary physical motions can be invaluable when choosing material to perform, creating new material, or preparing for a performance. For instance, if a passage has an increased dynamic or rhythmic density, it might be beneficial to incorporate more expressiveness into that part of the performance.
And for those of involved in music education, these insights offer new teaching strategies. By helping students to understand the correlation between physical movements and the perception of their audience, teachers can equip them to deliver more powerful and engaging performances.
Conclusion
It has been demonstrated, by other researchers, that a range of different external factors can have a positive or negative impact an audience’s perception of musical quality. For example, the age and economic profile of the audience, how the musicians dress and behave on stage, or their perceived attractiveness. Unfortunately, even the gender of performers can influence how a performance is received by an audience.
But motion capture technology provides a means of studying performer motion without the added complication of any of these external factors. And animating the motion data also allows for the manipulation of the visual elements of a performance without altering the original audio recording. What the research by Trevor and Huron indicates is that the physical motions of musicians can have a material impact on how audiences perceive the overall quality of a performance.
Playing music isn't just an auditory art form - it's a visual one too. Trevor and Huron’s work demonstrates that, from an audience’s perspective, ‘secondary motions’ can have a profound impact on perceived musicality and performance quality. A sway, a nod, or an expressive hand gesture - though seemingly trivial - might transform the audience's own experience of the music. It seems performance isn't only about the sound…..it's also about the show!
