Text-to-speech (TTS)
Text-to-speech models turn written text into spoken audio, and many can copy a voice from a short sample.
A TTS model takes text, and sometimes a short reference recording, and produces audio. Open models now cover dozens or even hundreds of languages, can stream audio while it is still being generated, and some can design a new voice from a written description. Streaming latency, the time until the first audio arrives, matters for voice assistants.
Check the license of the weights, not only the code: several of the best-known open TTS models are non-commercial only, and some projects limit use to research. Realistic voices can be misused, so some models add an inaudible watermark to their output.
Related terms
- Voice cloning Voice cloning makes a text-to-speech model speak in a particular person's voice, often from only a few seconds of recording.
- Speech recognition (ASR, speech-to-text) Speech recognition, also called ASR or speech-to-text, turns spoken audio into written text.
- Non-commercial license A non-commercial license lets you use a model for research, study and personal projects, but not to make money from it.