Minibeats

Mike Butera is an expert in many things. Sociology. Philosophy. Product design. Music. But the former professor and touring musician has always had one goal: to make it easier for everybody to engage with – and make – music. “We started from an idea,” Butera says. “What happens if music is inherently multisensory? Music has always combined sound with visual, tactile, and social elements. So, can we design new interfaces for music that capture all these dimensions using AR tools, and give them to everyone in an inherently playful way?” he says. That’s the spirit behind Artiphon, a company he founded that has released a series of popular digital instruments that make it easy to take music-making into your own hands. And it's also the inspiration for Minibeats, a new app from the company that uses AR and social media to unlock new types of musical expression. “Minibeats allows anyone to create interactive musical videos instantly. Our goal is to transform the world into your own personal Fantasia, where simple gestures become dynamic musical and visual experiences,” Butera says. “Augmented Reality enables us to use the camera as an instrument, letting you create cinematic moments in your everyday life.”

After joining Ghost, the Minibeats team wasted no time leveraging the tech they had at their disposal. “We learned about a lot of the new capabilities Snap had been working on in visual scanning, audio machine learning, and more,” Butera says. “We thought, what might we do with these? And that’s where Scan Band and the musical stickers came from.” 

Scan Band is a Lens that pulls together a number of innovations to create interactive, shareable musical moments—like AR “tattoos” that play unique snippets of music, with multiple ways to manipulate music using gestures. Scan Band was an instant success after debuting at Snap’s Partner Summit. And it laid the groundwork for later projects, which have seen them take on increasingly advanced technical challenges with support from the Ghost team. “The vision for Minibeats involved tackling a few challenges at once: capturing human gestures in real-time, translating those gestures into music, and designing responsive visual effects for more immersive videos. We explored many different technologies to accomplish this, and working with Snap and Lens Studio enabled us to design and develop each of these stages in the same environment,” they say. “Snap’s industry-leading AR technology is constantly evolving, and Minibeats has been able to use multiple emerging features such as AudioML and Scan to quickly deliver interactive musical experiences to a huge audience of social media users.”

They’ve learned a lot from the experience, which they’re applying to their next endeavor: a standalone Minibeats app with even more features, many of which were developed using Snap’s Camera Kit. “We are basically taking everything we've learned from the Snapchat Lenses and building an app that is like GarageBand in a camera,” Butera says. “It's a place you can go to have these immediate creative musical experiences. It’s an opportunity to see the world as your instrument, and make a soundtrack to your life, instantly, directly in video.”

For Butera, certain benefits of Lens Studio and Ghost make it an ideal platform and program for teams that want to develop in AR. Speed, for one. “Lens Studio offers unparalleled idea-to-prototype timing, letting us brainstorm and test concepts within days,” he says. Modular development is another. “With Camera Kit, we’ve been able to use Lens Studio to build experiences that live on Snapchat as well as in our via Camera Kit. Thus, instead of developing the same thing multiple times, we can build it once and deploy on multiple platforms.” Snapchat’s audience reach is perhaps most important to Butera’s democratic vision for musical expression. It’s helped him reach more people faster, test ideas quickly, and spread the joy of music to the world – which is why he does what he does in the first place. “The Snapchat audience is always hungry for new AR experiences, particularly those that blend music and visual effects. Snapchat is the best place to explore what’s possible with AR music,” he says.

Minibeats Snapcode