Robot uses deep learning to write and play its own music
ATLANTA, Georgia: Shimon, a four-armed, marimba playing robot, is writing and playing its own music using deep learning. This is the first of its two songs.
A marimba-playing robot with four arms and eight sticks is writing and playing its own compositions in a lab at the Georgia Institute of Technology. The pieces are generated using artificial intelligence and deep learning.
Researchers fed the robot nearly 5,000 complete songs — from Beethoven to the Beatles to Lady Gaga to Miles Davis — and more than two million motifs, riffs and licks of music. Aside from giving the machine a seed, or the first four measures to use as a starting point, no humans are involved in either the composition or the performance of the music.
The first two compositions are roughly 30 seconds in length. The robot, named Shimon, can be seen and heard playing them here and here.
Doctorate student Mason Bretan is the man behind the machine. He’s worked with Shimon for seven years, enabling it to “listen” to music played by humans and improvise over precomposed chord progressions. Now Shimon is a solo composer for the first time, generating the melody and harmonic structure on its own.
“Once Shimon learns the four measures we provide, it creates its own sequence of concepts and composes its own piece,” said Bretan, who will receive his doctorate in music technology this summer at Georgia Tech. “Shimon’s compositions represent how music sounds and looks when a robot uses deep neural networks to learn everything it knows about music from millions of human-made segments.”
Bretan says this is the first time a robot has used deep learning to create music. And unlike its days of improvising, when it played monophonically, Shimon is able to play harmonies and chords. It’s also thinking much more like a human musician, focusing less on the next note, as it did before, and more on the overall structure of the composition.
“When we play or listen to music, we don’t think about the next note and only that next note,” said Bretan.
“An artist has a bigger idea of what he or she is trying to achieve within the next few measures or later in the piece.
Shimon is now coming up with higher-level musical semantics. Rather than thinking note by note, it has a larger idea of what it wants to play as a whole.”
Shimon was created by Bretan’s advisor, Gil Weinberg, director of Georgia Tech’s Centre for Music Technology.
“This is a leap in Shimon’s musical quality because it’s using deep learning to create a more structured and coherent composition,” said Weinberg, a professor in the School of Music. “We want to explore whether robots could become musically creative and generate new music that we humans could find beautiful, inspiring and strange.” — Newswise
Shimon’s compositions represent how music sounds and looks when a robot uses deep neural networks to learn everything it knows about music from millions of human-made segments. – Mason Bretan, doctorate student