ElevenLabs, a company that makes AI audio tools, launched Instant Soundtracks on Oct. 15. A person uploads a video and the AI writes original music for it. ElevenLabs says the AI then looks through the footage for moments that call for a sound effect, makes those sounds and lines each one up with the moment on screen. ElevenLabs gives the example of a whoosh timed to the logo at the end of an ad.
With ElevenLabs' earlier music tool, which didn't look at the video, scoring a clip meant making the music, cutting it to fit the pictures, then making the sound effects separately and placing each one on an editing timeline, according to a news report. Instant Soundtracks is meant to handle those jobs together. It is built for short videos such as ads, and ElevenLabs says it times its sounds more tightly to cuts and on-screen action than before.
The music can also change with the scenes. ElevenLabs says a fast scene might get dance music or R&B, and the track might then move to acoustic guitar for a slower, more intimate one. People can also give the AI the video's own sound as a guide. ElevenLabs says it studies things such as the style, the beat and the instruments, then shapes a new score with the feel of the original.
The sound effects come out as their own layer, separate from the music, so people can move them, add more, swap the music or fix one moment without starting over. ElevenLabs says the tool is available now in its ElevenCreative editing app and on its ElevenMusic site, and developers can build it into their own apps.
How well it picks and times its sounds so far rests on ElevenLabs' own examples. The announcement does not say what the tool costs, which plans include it or what music the AI learned from. Developers need a paid plan, and film and TV use needs the business plan ElevenLabs calls Enterprise, according to a news report.