New Immersive Expressions - Composing Unique Classical Music with AFC

Since 2017, Yamaha has been collaborating with the Rochester Institute of Technology to explore the new possibilities in musical expression offered by its Active Field Control (AFC) immersive audio system. As part of the research, composer SiHyun Uhm was invited to create a new classical work which intertwined sound space and music.

Developed using the experience and knowledge gained from acoustic consulting and design of performance spaces since 1969, Yamaha Active Field Control (AFC) is a comprehensive sound space control system. This is achieved through two functions: AFC Enhance, which controls spatial resonance, and AFC Image, which controls the positions of sound sources.

Modern immersive sound systems are increasingly used in films, live pop concerts and installation art to create a heightened sense of realism for the audience. But there are very few examples of their use in classical music, despite the acoustic space long being considered a very important element in the genre.

Many classical Western composers have written music with the intention of creating a complete sound environment. These include the German composer Richard Wagner (1813-1883), who was involved in the design of the Bayreuth Festival Theatre, a theatre dedicated to the performance of his works.

Similarly, Yamaha believes variable immersive audio systems like AFC can bring significant benefits to content creators. Rather than just an extension of conventional electroacoustic systems, it’s possible that unprecedented musical expression could be created with the functions of AFC Image and AFC Enhance. Usually the acoustic space in a venue is fixed but, by offering a composer the option of controlling it, the potential for composition could be radically expanded.

Yamaha has pursued this research in collaboration with Associate Professor Sungyoung Kim of the Rochester Institute of Technology (RIT) in New York, USA. A former Yamaha employee, Professor Kim is an expert in the field of audio engineering, engaged in research on stereophonic rendering and subjective evaluation, digital archiving of auditory heritage and auditory training.

With no classical music specifically composed for performance using immersive sound systems, SiHyun Uhm - a composer at the Juilliard School of Music - was invited to compose a new piece which would use AFC as a key component. The composer challenged herself to express changes in time and place by using AFC to change the acoustics of the space in real time, along with sound effects which would be played back from different locations within the space.

For the composition process, an AFC system was built in a studio at RIT, allowing the composer to check the effectiveness of the system, matching the image of the music and the sound space, and exploring how to use the system most effectively.

String Quartet No. 2 For AFC System consisted of two distinct sections: a slow part representing the image of a mountain and a fast part representing the image of a desert. The music alternated between these two sections, culminating in a climax where they merged, reminiscent of a sci-fi film

It featured four AFC Enhance presets, respectively representing dry desert, plateau, ‘wind-blown’ and 'deep mountain’ soundscapes. AFC Image was used to position sound effects that represented elements of each scene, for example wind blowing and water dripping, helping to accurately depict each scene to the audience in the three-dimensional environment. The position, volume and movement of the sound effects was held as object audio information on the playback DAW and rendered in real-time.

By combining AFC Enhance and Image, SiHyun Uhm was able to convey not just the image of each landscape, but also more nuanced concepts such as darkness, tranquility, fear and a clear cloudless environment in the piece.

It was performed for the first time at the Yamaha Ginza Studio in Tokyo, Japan, in front of more than a hundred guests from various fields, including venue designers, sound designers, operators, composers and researchers.

For the performance, sheet music was used for the professional musicians as it would be in a normal classical piece. The music scores included notation of the four different AFC Enhance presets. These were marked 43 times in total, with 20 audio object information marks for AFC Image, creating a piece in which the immersive audio system was regarded as an integral instrument.

To synchronise the AFC system with the live performance, the sound engineer read the score during the performance and cued the AFC system at the appropriate times.

“I created a piece that goes back and forth between a fast section, inspired by the image of a desert, and a slower section inspired by the image of a mountain,” says SiHyun Uhm. “The two scenes had different characteristics, then at the end I tried to combine the spaces and merge them into one.

“Music without the system was an abstract description of the scene, but with the AFC system I could evoke the details, the specific place and even the time of day. The acoustic space created by AFC allowed me to create the atmosphere of the different places and to get deeper into the sound.”

The performance stimulated much discussion amongst the audience and created a lot of interest in the AFC system. One delegate said “I really enjoyed the unique compositional approach and real-time spatial concept of it as a classical piece. The composer's imagination of multiple singular acoustic spaces opened up many possibilicomposerties for use in music for opera and theatre.”