VoxCPM is a free and open-source text-to-speech (TTS) tool that converts text into lifelike speech without tokens, generating contextually appropriate and expressive audio that perfectly clons timbres in just 3–10 seconds of samples. You can download VoxCPM1.5 (800 million parameters) from Hugging Face, install it via pip, quickly synthesize speech using neat Python or CLI commands (RTF up to 0.15 on the RTX 4090), and fine-tune custom voices. With it, you can easily create natural-sounding audiobooks, podcasts, sound clones, or applications that sound at a professional level while saving time and money on voice production.
In the current field of speech synthesis, most systems still follow a similar technical path:
The speech is discretized into tokens (such as codec tokens), and then predicted using an autoregressive model, and finally reduced to a waveform.
VoxCPM tried to break this line.
It is not a simple “text-to-speech tool”, but a token-free continuous spatial speech modeling system. The core idea is that speech is essentially a continuous signal, and if we forcibly discretize it, we may lose the ability to express delicate acoustics.
From “discrete speech” to “continuous modeling”
Mainstream TTS models typically will:
- Compress speech into discrete tokens
- Predict token sequences with language models
- It is then decoded back to the audio
This method is mature in engineering, but there are two problems:
- Discrete tokens limit expression accuracy
- Semantic information and acoustic features are often coupled in the same sequence
VoxCPM does the following:
Model speech directly in a continuous space.
It does not rely on traditional discrete speech tokens but employs an end-to-end diffusion + autoregressive architecture to generate continuous speech representations directly from text.
The design goal is clear:
Make voice generation more natural, stable, and expressive.
Architecture design: a combination of diffusion + autoregression
VoxCPM uses a hybrid architecture:
- The upper layer uses autoregressive language modeling
- The lower layer uses a diffusion model for acoustic detail generation
This combination brings two advantages:
- Semantic stability – Autoregression is responsible for text understanding and structural modeling
- Acoustically detailed expression – Diffusion models excel at generating high-quality continuous signals
In addition, the model achieves some form of semantic-acoustic decoupling through hierarchical language modeling structures and FSQ constraints, making semantic control and timbre expression more clearly separated.
This is a relatively cutting-edge design on a technical level.
Two key capabilities
At the application level, VoxCPM highlights two capabilities:
Context-aware speech generation
Generative speech is not just about “reading the words”,
Instead, it generates expressions with changes in tone and rhythm according to the context.
In other words, it is closer to “reading aloud” than “broadcasting”.
Zero-shot voice cloning
With extremely short speech samples (seconds), the model can capture the speaker’s timbre characteristics and reproduce them on new text.
This means:
- Personalized voices can be generated quickly
- Mass customization training is not required
However, it should be emphasized:
This is a demonstration of model capabilities, not a lightweight commercial tool.
What positioning?
VoxCPM is more of a
A research-based project for unified phonetic-language modeling
Instead of:
- Commercial-grade TTS SaaS
- Ready-to-install speech synthesis tool
- Pure sound clone
Its value is more reflected in:
- Explore new routes for speech modeling
- Provides an experimental paradigm for continuous spatial modeling
- Pave the way for future voice multimodal large models
Difference from the mainstream route
Simple comparison:
| route | Mainstream model | VoxCPM |
|---|---|---|
| Modeling method | Discrete tokens | Continuous space |
| structure | Pure self-return | Autoregression + Diffusion |
| Semantic-acoustic | Strong coupling | Try decoupling |
| Positioning | The project is mature | Research and exploration |
Epilogue
VoxCPM is not your average TTS project.
It represents a rethinking of the speech generation paradigm.
If you pay attention to:
- Multimodal large model
- Phonetic-language unified modeling
- Application of diffusion models in the field of generation
Then it is worth reading deeply.
If you just want to find a simple TTS tool,
Then it may not be the lightest option.
Github：https://github.com/OpenBMB/VoxCPM
Tubing: