HeyGen, a company whose AI turns a written script into a video of a person talking, described HeyGen Voice in a research post on Oct. 14. It is a voice clone: an AI copy of one person's voice that can read out words the person never recorded. HeyGen says it needs only a one-minute recording made on a phone to learn how someone sounds, including their accent and the way they phrase things.
A recording of someone's voice also carries the microphone, the room and the noise around it, so a voice can sound exactly like the person and still sound wrong for the shot. HeyGen Voice keeps the person's voice but takes its cues for the rest from a picture of the scene and short written descriptions of it and of the delivery wanted. HeyGen says the model works out the room, how far away the microphone would be and the delivery the picture calls for. In HeyGen's showcase video, which pairs the voice with an AI-made video likeness of a man named Pavel, his voice moves from a studio to a beach, a factory floor and a moving car.
In HeyGen's app, the company says, HeyGen Voice comes with every video at no extra cost. Developers can build it into their own apps and pay for the amount of text it reads, and people who want a closer copy can pay a monthly fee to train it on longer recordings of their voice. HeyGen says a voice copied from that one recording can also speak Spanish, German, French or Chinese with a native accent.
A copied voice can be used to put words in someone's mouth. HeyGen says its platform requires verified consent from the person whose voice is copied and lets them ask for it to be removed, and that what people upload and make is subject to automated checks and human review. HeyGen says the speech its model makes carries an AI watermark, a hidden signal woven into the audio as it is made, which software can spot but people cannot hear. HeyGen has not said who can check for it.
How good it sounds rests mostly on HeyGen's own tests, in which its app users and outside testers preferred it to voices from ElevenLabs and Cartesia. On an outside firm's ranking, where listeners compare English voices copied from the same set of speakers, HeyGen Voice sits first, but so close to the runner-up that the two are effectively tied. That ranking judges the voice on its own, not how well it fits a scene, which so far rests on HeyGen's own reviewers. HeyGen's side-by-side demos use five scenes it chose, all with videos made by its own AI.