It can generate highly realistic virtual fitting images with more detailed details.
IDM-VTON is able to capture clothing details such as texture, pattern and stitching, which are accurately reproduced in fitting images.
Even in outdoor or complex background photos, this technology can accurately show the try-on effect of clothing, maintaining high quality image output.
When displaying the same costume on multiple different characters, IDM-VTON still maintains the consistency of the costume details.
This article considers an image-based virtual try-on, rendering an image of a person wearing a selected garment given a pair of images depicting the person and the garment respectively. Compared with other methods (such as GAN-based), previous works use existing sample-based repair diffusion models for virtual try-on to improve the naturalness of the generated visual effects, but they cannot retain the identity of the clothing.
To overcome this limitation, we propose a novel diffusion model that can improve clothing fidelity and generate realistic virtual try-on images.
Our method, called TON, uses two different modules to encode the semantics of clothing images; given the foundation of the diffusion model, UNet,
1) Integrate high-level semantics extracted from the visual encoder into the cross-attention layer, and then
2) Integrate low-level features extracted from parallel UNet into self-attention layers.
In addition, we also provide detailed text prompts for costumes and character images to enhance the authenticity of the generated visual effects. Finally, we propose a customization method using a pair of character costume images that significantly improves fidelity and authenticity.
Our experimental results show that our method outperforms previous methods (diffusion-based and GAN-based) in retaining clothing details and generating realistic virtual try-on images (both qualitatively and quantitatively). In addition, the proposed customization method proves its effectiveness in real-world scenarios. Our project page provides more visualizations.
If you want to learn more, you can click on the link below the video.
Thank you for watching this video. If you like it, please subscribe and like it. thank
Project address:https://idm-vton.github.io
Online experience:https://huggingface.co/spaces/yisol/IDM-VTON
Video: