Includes the complete text-to-video model training process, data processing, training details and model checkpoints.
The project was developed by the@YangYou1991 team, which is OpenAI Sora’s open source alternative for video generation.
You can generate 2 to 5 seconds of 512×512 video after just 3 days of training.
Open-Sora adopts a VAE model that is superior to the VQ-VAE in VideoGPT.
The project also explored different model architectures, including DiT (Diffusion Transformers), Latte, and the project’s proposed STDiT, to find the model architecture that best balances video quality and generation speed.
Supports inference using the official weights of DiT, Latte, and PixArt, allowing users to generate video from existing high-quality models.
It is a good thing to study…
GitHub：https://github.com/hpcaitech/Open-Sora
Video: