Changing tunes
How a classical composer’s innovative technology aims to revolutionise videogame soundtracks
Adaptive music has long played an important role in videogames, as far back as 1978’s Space Invaders, whose four simple but evocative notes increase in speed as your targets approach the bottom of the screen. For all the increased sophistication in musical fidelity since then, Jesper Nordin argues, the way music is created for videogames has remained fundamentally the same. While we expect every other aspect of a game, from lighting to physics, to be calculated and rendered in realtime, the music generally consists of recorded loops that have to be hard-coded into the game. Now Nordin, a contemporary classical composer, is hoping to change all that with Reactional Music.
Reactional is a rulebased system that allows composers and developers to set parameters that can change a piece of music note by note, adjusting pitch, tempo, intensity and more. It’s based on technology Nordin invented more than decade ago. “I always felt that the writing of a score, like writing on paper, is the second step for me,” he says. “I need to compose with my ears first, so I developed tools for my own compositional needs.”
The tech was first made available to others in 2018 in the form of mobile app Gestrument (a portmanteau of ‘gesture’ and ‘instrument’), which allowed musicians to create music in realtime using just their fingertips. The move into the videogame industry can be traced back to two conversations. The first was during a collaboration with a Swedish composer who works in the industry, who noted how useful this tech could be. The second came years later, with Kelly Sumner, former CEO of both Take-Two and Guitar Hero steward RedOctane – and now chairman of Reactional Music.
Reactional’s tech allows for closer syncing between the music and what is happening on screen. “If you look at big Hollywood movies, of course you expect things to happen in sync with the music, and if it doesn’t happen like that, it’s jarring and feels off,” Nordin says. “In games, it’s very tricky to get that matching in place if you haven’t hard-coded a specific game to a specific song.” Reactional’s two-way communication means that if a player defeats a boss, for instance, the music slows for a more dramatic beat rather than relying on a cued-in audio loop.
In practical terms, Reactionary allows a piece to be composed just once, with parameters then set and modified to fit different game states – raising the intensity during combat, for example, or mellowing in the menu screen, with all variants generated in realtime. “Composers would not have to bounce all these hundreds and hundreds of loops and then implement them into the game,” Nordin says.
As well as music composed specifically for games, Reactional can be used with licensed tracks, analysing the chord, pitch and rhythm data so that they sync in the same way. It’s a feat that recalls PSVR title Blood & Truth, which includes a number of needle drops during set pieces that dynamically mix with the game’s orchestral score – except that Reactional’s method isn’t bespoke, and works with any track in different contexts.
“It opens up the possibility for a player to swap out the music and still have the core sounds in the game designed by the studio and the composer, so you can start personalising your game with music in a way that hasn’t been possible before,” Nordin says. He sees this as a potential way to drive revenue in games, in a similar way to cosmetic items. To that end, in addition to courting composers and developers, Reactional has partnered with rights holders APM Music and Hipgnosis, opening up a catalogue stretching from Ed Sheeran to Nirvana. Even for developers who don’t intend to use licensed music in their game, Nordin points out, it’s entirely possible to use tracks as placeholder music. “You can just grab a Reactional theme, set some parameters, and start mapping things out.”
With Reactional’s beta released late last year, it’s still too early for Nordin to reveal the names of developers planning to use its technology in forthcoming projects. And it’s not suitable in every context. Diegetic music such as that played by GTA’s car radios doesn’t make sense to be synced with the game world, for example, while having the ability to freely swap out music raises questions around authorship. Still, any promise of a leap forward within an area that deserves the attention is tantalising.
If a player defeats a boss, the music slows for a more dramatic beat rather than relying on cued-in audio