DURHAM – Google Brain’s Magenta project, which is exploring the creative potential of machine learning (ML) and artificial intelligence (AI), has developed considerably since Google announced it at Moogfest three years ago. And, Magenta makes many of its ongoing developments available publicly online and collects feedback from musicians, artists and other users to advance the project.
Adam Roberts, senior software engineer and ML researcher discussed the nuts and bolts of Magenta at Moogfest over the weekend. Roberts, who did undergraduate work at the University of North Carolina at Chapel Hill, earned his PhD at Berkley, California, where he studied machine learning applied to genomics.
Google is developing both hardware and software to explore the potential of machine learning via its Magenta research, Roberts said. The project was officially announced at Moogfest three years ago. It has advanced considerably since then.
Google Brain’s second generation machine learning system Tensorflow, uses its Tensor processing unit (TPU), which Roberts told the Moogfest audience at the American Underground at Main in Durham, “Makes it much faster to train neural networks.” This AI accelerator was tailored to work with Tensorflow. Google says it delivers an order of magnitude better-optimized performance for machine learning.
Adam Roberts, a Google senior engineer, explaining the Magenta project at Moodfest. Copyright Capitol Cities. All rights reserved.
The deep learning systems using neural networks work differently than simple computer algorithms, Roberts explained. An ordinary computer can solve complex problems difficult for a human such as finding the shortest way to a destination by considering every possible route. “That would be time consuming for a person but a computer solves it with a simple algorithm,” Roberts said.
Giving a computer human skills
On the other hand, show a computer a picture and ask if it is a cat or a dog, a child could answer it, but the ordinary computer could not. “Skills we’re good at, computers are not, and visa versa,” Roberts said.
“Machine learning is one approach to helping computers achieve skills humans have but we can’t write simple algorithms for. You let the computer learn by showing it a bunch of examples. Deep learning uses connections between artificial neurons. You show it a lot of pictures of cats and dogs and it learns to generalize.”
You can’t actually look at the neural network and tell how it learned to tell the difference between images, “You can just tell that it had,” said Roberts. “Deep learning can be generative and create things. People believe it can create art in some sense.”
In music, for instance, you let computers learn the rules of music themselves from music. “You let it figure out how music works by looking at music. The object is not to mimic creativity but to augment and enhance it,” he said.
Google publishes all of its Magenta research code and tools to “Get them out in the wild,” Roberts said. “We engage with musicians and artists to get feedback and improve them.”
How do the tools work?
In music, Roberts explained, it uses the predictive power of neural networks to create. You give it a sequence of notes and ask it to predict what the next note will be and the next. Using different types of modeling adds additional structures such as key and rhythm. Google, he said, is “Far beyond” some of the tools available online now, “But you can still play with them.”
Moogfest audience listening to Google engineer Adam Roberts explaining part of the Google Magenta Project. Copyright Capitol Cities. All rights reserved.
To create sketches or paintings, neural networks use what is called “latent space learning,” which maps a data distribution. Examples of latent space models Magenta has developed include SketchRNN for sketches, NSynth for musical timbre, and MusicVAE a hierarchical recurrent variational autoencoder for learning latent spaces for musical scores.
Roberts pointed to the work of Tero Parviainen of creative.ai , who has built many apps using Magenta’s music models. HIs work helped inspire Google to develop an easy-to-use JavaScript API called Magenta.js,
