Fish Audio, a company that makes AI voices, released Drama 3 on Oct. 13. It is a model that turns written text into speech, and the company calls it its most emotive voice model yet, and says that means tears that sound real, anger held in check, a laugh breaking through a straight face. Users direct it the way they would brief an actor, writing a note in square brackets, such as [barely holding it together, forcing a smile], before the words it applies to.
A new note can go partway through a line, and Fish Audio says the voice changes right there, so a line can start breathless and drop to a terrified whisper. The company says sounds such as [shaky breath] or [small laugh] go exactly where they are written, and a tag can wrap just a few words to whisper or stress them. The company says the model speaks dozens of languages, and the notes don't have to be in the same language as the script. In Fish Audio's launch video, Uncle Roger directs an AI copy of his own voice, which the company says was made and used with his permission.
More feeling is of little use if the words come out wrong. On an outside testing firm's ranking of how well AI voices say tricky English words, such as brand and place names, numbers and email addresses, Drama 3, which Fish Audio still labels a preview, came first, with human reviewers judging about 93 in 100 of those words correct.
Drama 3 is available now on Fish Audio's website, in its phone apps for iPhone and Android, and for developers to build into their own products. The model's trained files are also published, so people can download it and run it on their own machines. Running it is free for research, for non-commercial use and for businesses that, with their affiliates, make less than $500,000 a year; bigger companies, and anyone offering it to others as a voice service, need a license from Fish Audio. The license also bans copying a real person's voice without their consent.
Fish Audio calls Drama 3 a preview that it will keep updating, says results vary from one try to the next, and suggests making a few and keeping the best. Two editing tools it showed, one that replays a line after feedback such as "softer than that one" and one that swaps a single word while keeping the rest of a recording, are only in small, invite-only tests. On the same firm's separate ranking of which voices listeners prefer, Drama 3 sits behind several rivals, and neither of the firm's rankings tests how well it acts out directions. Fish Audio calls it one large model, though only a small part of it works at each step, and its speed figures come from a powerful data-center chip, so it is not yet clear how well it runs on an ordinary computer.